[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:03 dist-test-slave-cbbz krb5kdc[3670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:03 dist-test-slave-cbbz krb5kdc[3670](info): set up 2 sockets
Aug 12 09:40:03 dist-test-slave-cbbz krb5kdc[3670](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 09:40:04.835675 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:04 dist-test-slave-cbbz krb5kdc[3670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:04.967545 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:42213
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:32963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:42213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:05.107100 3683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:05.107398 3683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:05.107456 3683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:05.111737 3683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:05.111802 3683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:05.111826 3683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:05.111846 3683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:05.111865 3683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:05.118036 3683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:32963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:42213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:42213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:05.119460 3683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:05.120774 3683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:05.135449 3690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:05.135499 3694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:05.136606 3683 server_base.cc:1034] running on GCE node
W20260812 09:40:05.137714 3689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:05.138408 3683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:05.140241 3683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:05.141412 3683 hybrid_clock.cc:648] HybridClock initialized: now 1786527605141398 us; error 51 us; skew 500 ppm
Aug 12 09:40:05 dist-test-slave-cbbz krb5kdc[3670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527605, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:05.145033 3683 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:05.146202 3683 webserver.cc:460] Webserver started at http://127.3.147.62:46761/ using document root <none> and password file <none>
I20260812 09:40:05.147154 3683 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:05.147224 3683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:05.147701 3683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:05.150745 3683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "64eb7fe7721a4d38bc1213609a852bb7"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "ff9177ff83301b16e8dcd9fa6d033be9"
server_key_iv: "80c92ae958c2236b89da62eff2f690da"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.151671 3683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "64eb7fe7721a4d38bc1213609a852bb7"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "ff9177ff83301b16e8dcd9fa6d033be9"
server_key_iv: "80c92ae958c2236b89da62eff2f690da"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.157183 3683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 09:40:05.162535 3701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:05.165257 3683 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260812 09:40:05.165534 3683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "64eb7fe7721a4d38bc1213609a852bb7"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "ff9177ff83301b16e8dcd9fa6d033be9"
server_key_iv: "80c92ae958c2236b89da62eff2f690da"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.165709 3683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:05.191095 3683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:05.196813 3683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:05.197273 3683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:05.220891 3683 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:42213
I20260812 09:40:05.221117 3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:42213 every 8 connection(s)
I20260812 09:40:05.222805 3683 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:05.226541 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 3683
I20260812 09:40:05.226706 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:05.228299 3660 external_mini_cluster.cc:1428] Setting key d5bb5dd5a91a313cc2f6f3d0472911c3
I20260812 09:40:05.230247 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:05.241922 3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7: Bootstrap starting.
I20260812 09:40:05.246796 3763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:05.248896 3763 log.cc:824] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:40:05 dist-test-slave-cbbz krb5kdc[3670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:05.251920 3763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7: No bootstrap required, opened a new log
I20260812 09:40:05.258369 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:05.237161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38546 (local address 127.3.147.62:42213)
0812 09:40:05.237731 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:05.237762 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:05.237840 (+ 78us) server_negotiation.cc:407] Connection header received
0812 09:40:05.240122 (+ 2282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:05.240197 (+ 75us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:05.240566 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:05.240890 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:05.243405 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.245108 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.246437 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.247094 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.247324 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:05.251820 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:05.251861 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:05.251875 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:05.251903 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:05.254341 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:05.254961 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:05.254970 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:05.254980 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:05.255098 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:05.255470 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:05.255474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:05.255476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:05.255971 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:05.256218 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:05.256854 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 09:40:05.257174 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":185,"threads_started":1}
I20260812 09:40:05.258886 3763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eb7fe7721a4d38bc1213609a852bb7" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42213 } }
I20260812 09:40:05.259274 3763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:05.259383 3763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64eb7fe7721a4d38bc1213609a852bb7, State: Initialized, Role: FOLLOWER
I20260812 09:40:05.260463 3763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eb7fe7721a4d38bc1213609a852bb7" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42213 } }
I20260812 09:40:05.260736 3763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:05.260897 3763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:05.261065 3763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:05.262849 3763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eb7fe7721a4d38bc1213609a852bb7" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42213 } }
I20260812 09:40:05.263502 3763 leader_election.cc:302] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64eb7fe7721a4d38bc1213609a852bb7; no voters:
I20260812 09:40:05.263983 3763 leader_election.cc:288] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:05.264148 3768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:05.264461 3768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [term 1 LEADER]: Becoming Leader. State: Replica: 64eb7fe7721a4d38bc1213609a852bb7, State: Running, Role: LEADER
I20260812 09:40:05.264994 3768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eb7fe7721a4d38bc1213609a852bb7" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42213 } }
I20260812 09:40:05.265712 3763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:05.267814 3770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64eb7fe7721a4d38bc1213609a852bb7. Latest consensus state: current_term: 1 leader_uuid: "64eb7fe7721a4d38bc1213609a852bb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eb7fe7721a4d38bc1213609a852bb7" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42213 } } }
I20260812 09:40:05.267962 3770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:05.267970 3769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64eb7fe7721a4d38bc1213609a852bb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eb7fe7721a4d38bc1213609a852bb7" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42213 } } }
I20260812 09:40:05.268491 3769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:05.268610 3778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:05.274184 3778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:05.283036 3778 catalog_manager.cc:1347] Generated new cluster ID: ca1d446c2daa4ae2b2c69abe7d465d44
I20260812 09:40:05.283188 3778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:05.302168 3778 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:05.303558 3778 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:05.334213 3778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64eb7fe7721a4d38bc1213609a852bb7: Generated new TSK 0
I20260812 09:40:05.335225 3778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:05.377302 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42213
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:32963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:05.547453 3790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:05.547781 3790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:05.547889 3790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:05.552315 3790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:05.552412 3790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:05.552531 3790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:05.558259 3790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:32963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:05.559435 3790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:05.560616 3790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:05.573415 3796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:05.573364 3798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:05.573424 3795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:05.573652 3790 server_base.cc:1034] running on GCE node
I20260812 09:40:05.574159 3790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:05.574844 3790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:05.576067 3790 hybrid_clock.cc:648] HybridClock initialized: now 1786527605576056 us; error 31 us; skew 500 ppm
Aug 12 09:40:05 dist-test-slave-cbbz krb5kdc[3670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527605, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:05.579711 3790 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:05.580713 3790 webserver.cc:460] Webserver started at http://127.3.147.1:41549/ using document root <none> and password file <none>
I20260812 09:40:05.581291 3790 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:05.581384 3790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:05.581633 3790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:05.583770 3790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "5a73a42e08a44aed834509ea829bc2b2"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "cd0b1df8b7d13fb640d3d9876b79b871"
server_key_iv: "09a03c7f7a2ca16a61a461b4661abcce"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.584403 3790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "5a73a42e08a44aed834509ea829bc2b2"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "cd0b1df8b7d13fb640d3d9876b79b871"
server_key_iv: "09a03c7f7a2ca16a61a461b4661abcce"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.588464 3790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 09:40:05.591028 3807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:05.592262 3790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:05.592392 3790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "5a73a42e08a44aed834509ea829bc2b2"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "cd0b1df8b7d13fb640d3d9876b79b871"
server_key_iv: "09a03c7f7a2ca16a61a461b4661abcce"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.592504 3790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:05.623747 3790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:05.627593 3790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:05.627874 3790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:05.628865 3790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:05.630069 3790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:05.630156 3790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:05.630236 3790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:05.630293 3790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:05.655838 3790 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:39459
I20260812 09:40:05.656179 3937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:39459 every 8 connection(s)
I20260812 09:40:05.657063 3790 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:05.665977 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 3790
I20260812 09:40:05.666102 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:05.666430 3660 external_mini_cluster.cc:1428] Setting key e72137d29dfb159c6af9f3ad4153925b
Aug 12 09:40:05 dist-test-slave-cbbz krb5kdc[3670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527605, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:05.676429 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:05.661390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:44967 (local address 127.3.147.62:42213)
0812 09:40:05.661533 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:05.661545 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:05.662640 (+ 1095us) server_negotiation.cc:407] Connection header received
0812 09:40:05.664549 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:05.664553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:05.664602 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:05.664681 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:05.666359 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.667150 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.668854 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.669186 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.669299 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:05.672124 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:05.672171 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:05.672175 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:05.672209 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:05.673755 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:05.674471 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:05.674475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:05.674477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:05.674601 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:05.675071 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:05.675073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:05.675075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:05.675249 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:05.675349 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:05.676028 (+ 679us) server_negotiation.cc:299] Negotiation successful
0812 09:40:05.676260 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:40:05.677881 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:05.661814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42213 (local address 127.3.147.1:44967)
0812 09:40:05.662475 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:05.662522 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:05.664348 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:05.664894 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:05.664900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:05.665372 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:05.666185 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:05.666207 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.667288 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.667292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:05.668732 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:05.668742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.669339 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.669343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:05.669576 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:05.670281 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:05.670298 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:05.671954 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:05.673872 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:05.673881 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:05.673920 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:05.674340 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:05.674738 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:05.674744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:05.674748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:05.674964 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:05.675370 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:05.675380 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:05.675722 (+ 342us) client_negotiation.cc:769] Sending connection context
0812 09:40:05.675962 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 09:40:05.676381 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":182,"threads_started":1}
I20260812 09:40:05.679383 3938 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42213
I20260812 09:40:05.679644 3938 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:05.680208 3938 heartbeater.cc:511] Master 127.3.147.62:42213 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:05.682245 3722 ts_manager.cc:194] Registered new tserver with Master: 5a73a42e08a44aed834509ea829bc2b2 (127.3.147.1:39459)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:40:05.683630 3722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:44967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:05.702983 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42213
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:32963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:05.870759 3943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:05.871069 3943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:05.871165 3943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:05.875547 3943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:05.875718 3943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:05.875883 3943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:05.882033 3943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:32963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:05.883225 3943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:05.884316 3943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:05.895516 3949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:05.895550 3951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:05.895462 3948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:05.895709 3943 server_base.cc:1034] running on GCE node
I20260812 09:40:05.896245 3943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:05.896979 3943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:05.898173 3943 hybrid_clock.cc:648] HybridClock initialized: now 1786527605898142 us; error 48 us; skew 500 ppm
Aug 12 09:40:05 dist-test-slave-cbbz krb5kdc[3670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527605, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:05.901836 3943 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:05.902776 3943 webserver.cc:460] Webserver started at http://127.3.147.2:41237/ using document root <none> and password file <none>
I20260812 09:40:05.903363 3943 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:05.903460 3943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:05.903702 3943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:05.906067 3943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "26e914b2cac14251ba611fddf31d5bd3"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "be79830ad552412cb05453dd69ba669d"
server_key_iv: "9e0eaa12e5f2d34f9b0d102318970466"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.906700 3943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "26e914b2cac14251ba611fddf31d5bd3"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "be79830ad552412cb05453dd69ba669d"
server_key_iv: "9e0eaa12e5f2d34f9b0d102318970466"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.910892 3943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 09:40:05.913537 3958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:05.914740 3943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:05.914891 3943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "26e914b2cac14251ba611fddf31d5bd3"
format_stamp: "Formatted at 2026-08-12 09:40:05 on dist-test-slave-cbbz"
server_key: "be79830ad552412cb05453dd69ba669d"
server_key_iv: "9e0eaa12e5f2d34f9b0d102318970466"
server_key_version: "encryptionkey@0"
I20260812 09:40:05.915002 3943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:05.933223 3943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:05.936466 3943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:05.936724 3943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:05.937613 3943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:05.938627 3943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:05.938700 3943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:05.938778 3943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:05.938830 3943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:05.963814 3943 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46519
I20260812 09:40:05.963909 4086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46519 every 8 connection(s)
I20260812 09:40:05.964892 3943 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:05.972858 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 3943
I20260812 09:40:05.972996 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:05.973333 3660 external_mini_cluster.cc:1428] Setting key 9453a920ff786b069a7e79f743904cb7
Aug 12 09:40:05 dist-test-slave-cbbz krb5kdc[3670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527605, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:05.982252 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:05.967096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:37409 (local address 127.3.147.62:42213)
0812 09:40:05.967255 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:05.967258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:05.968485 (+ 1227us) server_negotiation.cc:407] Connection header received
0812 09:40:05.970236 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:05.970239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:05.970289 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:05.970410 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:05.971912 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.972444 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.974219 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.974520 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.974644 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:05.978381 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:05.978385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:05.978387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:05.978414 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:05.979918 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:05.980544 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:05.980548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:05.980550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:05.980640 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:05.981025 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:05.981029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:05.981030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:05.981278 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:05.981396 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:05.981954 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 09:40:05.982083 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 09:40:05.983238 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:05.967502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42213 (local address 127.3.147.2:37409)
0812 09:40:05.968325 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:05.968369 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:05.970013 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:05.970616 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:05.970623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:05.971141 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:05.971752 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:05.971769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.972634 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.972639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:05.974084 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:05.974095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:05.974712 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:05.974717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:05.974974 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:05.975748 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:05.975776 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:05.978176 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:05.980054 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:05.980060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:05.980078 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:05.980429 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:05.980742 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:05.980746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:05.980748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:05.980914 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:05.981396 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:05.981404 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:05.981681 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 09:40:05.981897 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 09:40:05.982253 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":266,"threads_started":1}
I20260812 09:40:05.985076 4087 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42213
I20260812 09:40:05.985379 4087 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:05.985944 4087 heartbeater.cc:511] Master 127.3.147.62:42213 requested a full tablet report, sending...
I20260812 09:40:05.987111 3722 ts_manager.cc:194] Registered new tserver with Master: 26e914b2cac14251ba611fddf31d5bd3 (127.3.147.2:46519)
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:40:05.987784 3722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:37409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:06.010912 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42213
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:32963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:06.180800 4092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:06.181141 4092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:06.181258 4092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:06.185730 4092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:06.185829 4092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:06.185923 4092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:06.191903 4092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:32963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42213
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:06.193137 4092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:06.194254 4092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:06.206427 4098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:06.206431 4101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:06.206373 4099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:06.206681 4092 server_base.cc:1034] running on GCE node
I20260812 09:40:06.207147 4092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:06.207831 4092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:06.209038 4092 hybrid_clock.cc:648] HybridClock initialized: now 1786527606209006 us; error 52 us; skew 500 ppm
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[3670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527606, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:06.212469 4092 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:06.213599 4092 webserver.cc:460] Webserver started at http://127.3.147.3:43169/ using document root <none> and password file <none>
I20260812 09:40:06.214190 4092 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:06.214289 4092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:06.214542 4092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:06.217010 4092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "109725667ba14638a233b189db9bca15"
format_stamp: "Formatted at 2026-08-12 09:40:06 on dist-test-slave-cbbz"
server_key: "a725d9cbec62de7de00549f9d1ca85e3"
server_key_iv: "fb04f012b4aba9a7e6edf8b84edfed97"
server_key_version: "encryptionkey@0"
I20260812 09:40:06.217656 4092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "109725667ba14638a233b189db9bca15"
format_stamp: "Formatted at 2026-08-12 09:40:06 on dist-test-slave-cbbz"
server_key: "a725d9cbec62de7de00549f9d1ca85e3"
server_key_iv: "fb04f012b4aba9a7e6edf8b84edfed97"
server_key_version: "encryptionkey@0"
I20260812 09:40:06.222237 4092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:40:06.225055 4108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:06.226398 4092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:06.226526 4092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "109725667ba14638a233b189db9bca15"
format_stamp: "Formatted at 2026-08-12 09:40:06 on dist-test-slave-cbbz"
server_key: "a725d9cbec62de7de00549f9d1ca85e3"
server_key_iv: "fb04f012b4aba9a7e6edf8b84edfed97"
server_key_version: "encryptionkey@0"
I20260812 09:40:06.226632 4092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:06.247686 4092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:06.250948 4092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:06.251184 4092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:06.251976 4092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:06.253049 4092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:06.253098 4092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:06.253165 4092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:06.253197 4092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:06.278640 4092 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:44015
I20260812 09:40:06.278748 4232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:44015 every 8 connection(s)
I20260812 09:40:06.279734 4092 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:06.289419 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 4092
I20260812 09:40:06.289559 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:06.289908 3660 external_mini_cluster.cc:1428] Setting key 8d0ff3e1c648f457ca2f63d3fbe0afc9
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[3670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527606, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:06.296578 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:06.282160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:54211 (local address 127.3.147.62:42213)
0812 09:40:06.282307 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:06.282310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:06.283384 (+ 1074us) server_negotiation.cc:407] Connection header received
0812 09:40:06.285383 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:06.285393 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:06.285463 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:06.285593 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:06.287191 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:06.287741 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:06.289202 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:06.289443 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:06.289579 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:06.293167 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:06.293171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:06.293174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:06.293204 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:06.294560 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:06.295045 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:06.295048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:06.295049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:06.295136 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:06.295456 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:06.295459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:06.295460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:06.295670 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:06.295767 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:06.296296 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 09:40:06.296435 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 09:40:06.297535 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:06.282558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42213 (local address 127.3.147.3:54211)
0812 09:40:06.283220 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:06.283265 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:06.284979 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:06.285792 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:06.285799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:06.286380 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:06.287010 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:06.287027 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:06.287888 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:06.287897 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:06.289057 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:06.289069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:06.289657 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:06.289664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:06.289969 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:06.290911 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:06.290935 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:06.292970 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:06.294673 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:06.294678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:06.294688 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:06.294942 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:06.295218 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:06.295221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:06.295222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:06.295365 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:06.295782 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:06.295787 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:06.296007 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 09:40:06.296243 (+ 236us) client_negotiation.cc:240] Negotiation successful
0812 09:40:06.296501 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":185,"threads_started":1}
I20260812 09:40:06.298964 4233 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42213
I20260812 09:40:06.299424 4233 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:06.300194 4233 heartbeater.cc:511] Master 127.3.147.62:42213 requested a full tablet report, sending...
I20260812 09:40:06.301544 3722 ts_manager.cc:194] Registered new tserver with Master: 109725667ba14638a233b189db9bca15 (127.3.147.3:44015)
I20260812 09:40:06.302450 3722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:54211
I20260812 09:40:06.305146 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[3670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527606, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[3670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527606, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:06.336685 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:06.326578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.3.147.1:39459)
0812 09:40:06.326928 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:06.326938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:06.326967 (+ 29us) server_negotiation.cc:407] Connection header received
0812 09:40:06.327019 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:06.327024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:06.327214 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:06.327333 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:06.328110 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:06.328911 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:06.329902 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:06.330282 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:06.330443 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:06.333274 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:06.333284 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:06.333296 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:06.333320 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:06.334982 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:06.335395 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:06.335400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:06.335401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:06.335482 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:06.335716 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:06.335718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:06.335719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:06.335975 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:06.336191 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:06.336409 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 09:40:06.336537 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":152,"threads_started":1}
W20260812 09:40:06.337697 3839 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48980
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[3670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[3670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:06.369758 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:06.360494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48982 (local address 127.3.147.1:39459)
0812 09:40:06.360681 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:06.360687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:06.360829 (+ 142us) server_negotiation.cc:407] Connection header received
0812 09:40:06.361017 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:06.361022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:06.361074 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:06.361165 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:06.361913 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:06.362469 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:06.363383 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:06.363620 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:06.363729 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:06.366453 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:06.366462 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:06.366465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:06.366490 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:06.367920 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:06.368424 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:06.368428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:06.368429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:06.368512 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:06.368839 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:06.368843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:06.368844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:06.369015 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:06.369351 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:06.369460 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 09:40:06.369586 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 09:40:06.370962 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 3790
I20260812 09:40:06.382298 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 3943
I20260812 09:40:06.393098 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 4092
I20260812 09:40:06.400408 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 3683
2026-08-12T09:40:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2722 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[4248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[4248](info): set up 2 sockets
Aug 12 09:40:06 dist-test-slave-cbbz krb5kdc[4248](info): commencing operation
krb5kdc: starting...
W20260812 09:40:07.521799 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:07 dist-test-slave-cbbz krb5kdc[4248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:07.646989 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43907
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:43907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:07.758358 4262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:07.758651 4262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:07.758736 4262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:07.762761 4262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:07.762845 4262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:07.762887 4262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:07.762944 4262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:07.762985 4262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:07.768186 4262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:43907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:07.769205 4262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:07.770206 4262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:07.778538 4273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:07.778532 4269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:07.778543 4270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:07.778973 4262 server_base.cc:1034] running on GCE node
I20260812 09:40:07.779592 4262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:07.780826 4262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:07.782021 4262 hybrid_clock.cc:648] HybridClock initialized: now 1786527607781992 us; error 44 us; skew 500 ppm
Aug 12 09:40:07 dist-test-slave-cbbz krb5kdc[4248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527607, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:07.785043 4262 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:07.786007 4262 webserver.cc:460] Webserver started at http://127.3.147.62:46605/ using document root <none> and password file <none>
I20260812 09:40:07.786602 4262 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:07.786696 4262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:07.786943 4262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:07.789131 4262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "5cec8ba3b4384795bcd052774f1c0a82"
format_stamp: "Formatted at 2026-08-12 09:40:07 on dist-test-slave-cbbz"
server_key: "88d09c8118fc2c4e8fdc40a58732f07b"
server_key_iv: "0105aac415671d35aad3a262b3e0b106"
server_key_version: "encryptionkey@0"
I20260812 09:40:07.789713 4262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "5cec8ba3b4384795bcd052774f1c0a82"
format_stamp: "Formatted at 2026-08-12 09:40:07 on dist-test-slave-cbbz"
server_key: "88d09c8118fc2c4e8fdc40a58732f07b"
server_key_iv: "0105aac415671d35aad3a262b3e0b106"
server_key_version: "encryptionkey@0"
I20260812 09:40:07.793598 4262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 09:40:07.796020 4280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:07.797330 4262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:07.797475 4262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "5cec8ba3b4384795bcd052774f1c0a82"
format_stamp: "Formatted at 2026-08-12 09:40:07 on dist-test-slave-cbbz"
server_key: "88d09c8118fc2c4e8fdc40a58732f07b"
server_key_iv: "0105aac415671d35aad3a262b3e0b106"
server_key_version: "encryptionkey@0"
I20260812 09:40:07.797590 4262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:07.815227 4262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:07.818403 4262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:07.818621 4262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:07.826251 4262 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:43907
I20260812 09:40:07.826273 4340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:43907 every 8 connection(s)
I20260812 09:40:07.827225 4262 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:07.830061 4341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:07.833655 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 4262
I20260812 09:40:07.833794 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:07.834046 3660 external_mini_cluster.cc:1428] Setting key a2fab6ab32d60664a5f66a8fad18da51
I20260812 09:40:07.836058 4341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82: Bootstrap starting.
I20260812 09:40:07.838661 4341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:07.839519 4341 log.cc:824] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:07.841732 4341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82: No bootstrap required, opened a new log
Aug 12 09:40:07 dist-test-slave-cbbz krb5kdc[4248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:07.845836 4341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cec8ba3b4384795bcd052774f1c0a82" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43907 } }
I20260812 09:40:07.846088 4341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:07.846174 4341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cec8ba3b4384795bcd052774f1c0a82, State: Initialized, Role: FOLLOWER
I20260812 09:40:07.846674 4341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cec8ba3b4384795bcd052774f1c0a82" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43907 } }
I20260812 09:40:07.846833 4341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:07.846925 4341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:07.847034 4341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:07.848093 4341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cec8ba3b4384795bcd052774f1c0a82" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43907 } }
I20260812 09:40:07.848500 4341 leader_election.cc:302] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cec8ba3b4384795bcd052774f1c0a82; no voters:
I20260812 09:40:07.848556 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:07.835387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.3.147.62:43907)
0812 09:40:07.835791 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:07.835807 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:07.835842 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:40:07.836541 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:07.836560 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:07.836887 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:07.837168 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:07.838067 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:07.839109 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:07.840258 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:07.840726 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:07.840849 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:07.843641 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:07.843646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:07.843659 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:07.843685 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:07.845513 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:07.845975 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:07.845981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:07.845985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:07.846122 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:07.846461 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:07.846468 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:07.846470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:07.846897 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:07.847157 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:07.847376 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 09:40:07.847679 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":128,"threads_started":1}
I20260812 09:40:07.848825 4341 leader_election.cc:288] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:07.848912 4346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:07.849102 4346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [term 1 LEADER]: Becoming Leader. State: Replica: 5cec8ba3b4384795bcd052774f1c0a82, State: Running, Role: LEADER
I20260812 09:40:07.849386 4346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cec8ba3b4384795bcd052774f1c0a82" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43907 } }
I20260812 09:40:07.850147 4341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:07.851234 4347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cec8ba3b4384795bcd052774f1c0a82. Latest consensus state: current_term: 1 leader_uuid: "5cec8ba3b4384795bcd052774f1c0a82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cec8ba3b4384795bcd052774f1c0a82" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43907 } } }
I20260812 09:40:07.851369 4347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:07.851727 4348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cec8ba3b4384795bcd052774f1c0a82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cec8ba3b4384795bcd052774f1c0a82" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43907 } } }
I20260812 09:40:07.851837 4356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:07.851848 4348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:07.855157 4356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:07.860572 4356 catalog_manager.cc:1347] Generated new cluster ID: 1e85cc0da4d246119a622b5b09193192
I20260812 09:40:07.860651 4356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:07.871769 4356 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:07.873014 4356 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:07.882956 4356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cec8ba3b4384795bcd052774f1c0a82: Generated new TSK 0
I20260812 09:40:07.883683 4356 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:07.917088 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43907
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:08.030704 4368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:08.030949 4368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:08.030997 4368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:08.034893 4368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:08.034940 4368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:08.034994 4368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:08.040050 4368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43907
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:08.041020 4368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:08.041989 4368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:08.051630 4373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:08.051692 4374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:08.051652 4376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:08.052047 4368 server_base.cc:1034] running on GCE node
I20260812 09:40:08.052484 4368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:08.053105 4368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:08.054291 4368 hybrid_clock.cc:648] HybridClock initialized: now 1786527608054270 us; error 30 us; skew 500 ppm
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:08.057281 4368 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:08.058094 4368 webserver.cc:460] Webserver started at http://127.3.147.1:42937/ using document root <none> and password file <none>
I20260812 09:40:08.058660 4368 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:08.058754 4368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:08.058997 4368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:08.061064 4368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "c6cad435801a4cd197ef71a2d137e601"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "77c270775b3eac659840c8c51d68799f"
server_key_iv: "950290dcb8da804fc0ea6a2d89a517bb"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.061632 4368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "c6cad435801a4cd197ef71a2d137e601"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "77c270775b3eac659840c8c51d68799f"
server_key_iv: "950290dcb8da804fc0ea6a2d89a517bb"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.065327 4368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:08.067677 4385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.068867 4368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:08.069011 4368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "c6cad435801a4cd197ef71a2d137e601"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "77c270775b3eac659840c8c51d68799f"
server_key_iv: "950290dcb8da804fc0ea6a2d89a517bb"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.069118 4368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:08.080006 4368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:08.082770 4368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:08.082966 4368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:08.083508 4368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:08.084391 4368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:08.084470 4368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.084537 4368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:08.084579 4368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.093681 4368 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:36265
I20260812 09:40:08.093732 4517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:36265 every 8 connection(s)
I20260812 09:40:08.094545 4368 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:08.103389 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 4368
I20260812 09:40:08.103510 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:08.103817 3660 external_mini_cluster.cc:1428] Setting key 5de85a5d7114864fb26ae2ef374253b5
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:08.110970 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.096533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:32993 (local address 127.3.147.62:43907)
0812 09:40:08.096666 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.096670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.097488 (+ 818us) server_negotiation.cc:407] Connection header received
0812 09:40:08.098865 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.098869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.098930 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.099040 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:08.100210 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.100760 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.103172 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.103542 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.103664 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.107246 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:08.107250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:08.107253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:08.107286 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:08.108883 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.109417 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.109421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.109423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.109510 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.109867 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.109872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.109874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.110061 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:08.110175 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.110664 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.110792 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:40:08.111737 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.096890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43907 (local address 127.3.147.1:32993)
0812 09:40:08.097340 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:08.097383 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:08.098632 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:08.099212 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:08.099219 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:08.099572 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:08.100028 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.100040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.100910 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.100917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.103004 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.103027 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.103785 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.103789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.104080 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.104959 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:08.104975 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:08.107069 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:08.109011 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:08.109016 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:08.109031 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:08.109307 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:08.109622 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:08.109625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:08.109627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:08.109767 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:08.110179 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:08.110187 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:08.110424 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 09:40:08.110612 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 09:40:08.110931 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":108,"threads_started":1}
I20260812 09:40:08.113013 4518 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43907
I20260812 09:40:08.113317 4518 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:08.113931 4518 heartbeater.cc:511] Master 127.3.147.62:43907 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:08.115578 4297 ts_manager.cc:194] Registered new tserver with Master: c6cad435801a4cd197ef71a2d137e601 (127.3.147.1:36265)
I20260812 09:40:08.116879 4297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:32993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:08.133213 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43907
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:08.245707 4523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:08.245966 4523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:08.246009 4523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:08.250058 4523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:08.250123 4523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:08.250185 4523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:08.255445 4523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43907
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:08.256459 4523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:08.257417 4523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:08.267100 4529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:08.267069 4532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:08.267088 4528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:08.267547 4523 server_base.cc:1034] running on GCE node
I20260812 09:40:08.267943 4523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:08.268596 4523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:08.269757 4523 hybrid_clock.cc:648] HybridClock initialized: now 1786527608269720 us; error 48 us; skew 500 ppm
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:08.272541 4523 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:08.273494 4523 webserver.cc:460] Webserver started at http://127.3.147.2:37725/ using document root <none> and password file <none>
I20260812 09:40:08.274083 4523 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:08.274179 4523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:08.274417 4523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:08.276450 4523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "64a0971c477146948cdf570f70a18fe3"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "8ec39246a1d1a994f6f7350f7d3cbde0"
server_key_iv: "1b2af8412ea1ca0b02907bb51f61e4e7"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.277031 4523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "64a0971c477146948cdf570f70a18fe3"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "8ec39246a1d1a994f6f7350f7d3cbde0"
server_key_iv: "1b2af8412ea1ca0b02907bb51f61e4e7"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.280606 4523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:40:08.283099 4540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.284443 4523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:08.284578 4523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "64a0971c477146948cdf570f70a18fe3"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "8ec39246a1d1a994f6f7350f7d3cbde0"
server_key_iv: "1b2af8412ea1ca0b02907bb51f61e4e7"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.284695 4523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:08.297992 4523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:08.301144 4523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:08.301399 4523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:08.302006 4523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:08.302793 4523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:08.302863 4523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.302934 4523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:08.302978 4523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.313087 4523 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:41461
I20260812 09:40:08.313158 4674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:41461 every 8 connection(s)
I20260812 09:40:08.313977 4523 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:08.320322 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 4523
I20260812 09:40:08.320459 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:08.320811 3660 external_mini_cluster.cc:1428] Setting key a4e9b86c8bfb83bedcdd1f25571697ca
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:08.328104 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.315733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:56051 (local address 127.3.147.62:43907)
0812 09:40:08.315865 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.315872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.316743 (+ 871us) server_negotiation.cc:407] Connection header received
0812 09:40:08.318007 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.318010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.318059 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.318135 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:08.319281 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.319807 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.321102 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.321409 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.321510 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.324288 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:08.324291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:08.324293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:08.324315 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:08.325816 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.326441 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.326444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.326445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.326527 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.326931 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.326933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.326935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.327111 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:08.327187 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.327861 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.327966 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 09:40:08.329286 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.316082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43907 (local address 127.3.147.2:56051)
0812 09:40:08.316590 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:08.316632 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:08.317819 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:08.318309 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:08.318315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:08.318703 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:08.319146 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.319159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.319914 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.319916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.320981 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.320991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.321588 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.321592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.321785 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.322433 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:08.322449 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:08.324059 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:08.325937 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:08.325945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:08.325962 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:08.326317 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:08.326632 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:08.326638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:08.326642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:08.326828 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:08.327223 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:08.327233 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:08.327559 (+ 326us) client_negotiation.cc:769] Sending connection context
0812 09:40:08.327789 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 09:40:08.328231 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":124,"threads_started":1}
I20260812 09:40:08.330525 4675 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43907
I20260812 09:40:08.330787 4675 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:08.331274 4675 heartbeater.cc:511] Master 127.3.147.62:43907 requested a full tablet report, sending...
I20260812 09:40:08.332367 4297 ts_manager.cc:194] Registered new tserver with Master: 64a0971c477146948cdf570f70a18fe3 (127.3.147.2:41461)
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:40:08.332955 4297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:56051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:08.349205 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43907
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:08.460770 4682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:08.461035 4682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:08.461133 4682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:08.465364 4682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:08.465441 4682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:08.465525 4682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:08.470685 4682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43907
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:08.471691 4682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:08.472765 4682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:08.482409 4687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:08.482390 4690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:08.482393 4688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:08.482638 4682 server_base.cc:1034] running on GCE node
I20260812 09:40:08.483173 4682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:08.483799 4682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:08.484992 4682 hybrid_clock.cc:648] HybridClock initialized: now 1786527608484977 us; error 36 us; skew 500 ppm
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:08.487778 4682 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:08.488816 4682 webserver.cc:460] Webserver started at http://127.3.147.3:45861/ using document root <none> and password file <none>
I20260812 09:40:08.489375 4682 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:08.489467 4682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:08.489713 4682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:08.491714 4682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "72a6a2f3b00d472daca48bcf1611f267"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "e149d2d8940e5f5fdeb21bd1aeb6334c"
server_key_iv: "a24b165acedde23dd0aac4d11d10acaa"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.492331 4682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "72a6a2f3b00d472daca48bcf1611f267"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "e149d2d8940e5f5fdeb21bd1aeb6334c"
server_key_iv: "a24b165acedde23dd0aac4d11d10acaa"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.495958 4682 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:08.498354 4697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.499460 4682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:08.499601 4682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "72a6a2f3b00d472daca48bcf1611f267"
format_stamp: "Formatted at 2026-08-12 09:40:08 on dist-test-slave-cbbz"
server_key: "e149d2d8940e5f5fdeb21bd1aeb6334c"
server_key_iv: "a24b165acedde23dd0aac4d11d10acaa"
server_key_version: "encryptionkey@0"
I20260812 09:40:08.499711 4682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:08.529441 4682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:08.532565 4682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:08.532789 4682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:08.533424 4682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:08.534303 4682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:08.534377 4682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.534451 4682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:08.534505 4682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:08.544564 4682 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:43219
I20260812 09:40:08.544577 4828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:43219 every 8 connection(s)
I20260812 09:40:08.545536 4682 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:08.555796 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 4682
I20260812 09:40:08.555972 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:08.556418 3660 external_mini_cluster.cc:1428] Setting key cb63f8f2be247575f49831fb849c1966
I20260812 09:40:08.559803 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.547424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:34845 (local address 127.3.147.62:43907)
0812 09:40:08.547600 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.547605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.548499 (+ 894us) server_negotiation.cc:407] Connection header received
0812 09:40:08.549845 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.549848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.549898 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.549973 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:08.551338 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.551903 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.553155 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.553376 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.553464 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.555934 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:08.555941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:08.555945 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:08.555979 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:08.557736 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.558264 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.558267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.558269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.558361 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.558722 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.558730 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.558732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.558986 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:08.559080 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.559581 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.559686 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 09:40:08.560647 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.547813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43907 (local address 127.3.147.3:34845)
0812 09:40:08.548351 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:08.548394 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:08.549607 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:08.550191 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:08.550197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:08.550547 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:08.551191 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.551210 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.552046 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.552058 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.553024 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.553034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.553514 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.553517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.553655 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.554235 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:08.554250 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:08.555737 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:08.557862 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:08.557868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:08.557884 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:08.558150 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:08.558455 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:08.558459 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:08.558461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:08.558598 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:08.559095 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:08.559100 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:08.559338 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 09:40:08.559496 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 09:40:08.559739 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":156,"threads_started":1}
I20260812 09:40:08.561781 4829 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43907
I20260812 09:40:08.562028 4829 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:08.562520 4829 heartbeater.cc:511] Master 127.3.147.62:43907 requested a full tablet report, sending...
I20260812 09:40:08.563596 4297 ts_manager.cc:194] Registered new tserver with Master: 72a6a2f3b00d472daca48bcf1611f267 (127.3.147.3:43219)
I20260812 09:40:08.564258 4297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:34845
I20260812 09:40:08.570504 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:40:08.582466 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.574993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.3.147.62:43907)
0812 09:40:08.575111 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.575115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.575371 (+ 256us) server_negotiation.cc:407] Connection header received
0812 09:40:08.575456 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.575458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.575500 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.575567 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:08.576407 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.576899 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.577681 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.577879 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.577953 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.579396 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:08.579399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:08.579401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:08.579423 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:08.580875 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.581308 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.581311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.581312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.581396 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.581651 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.581653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.581654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.581884 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:08.581987 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.582231 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.582353 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 09:40:08.588068 4297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:08.591401 4297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:08.607671 4842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.601273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:41461 (local address 127.0.0.1:48186)
0812 09:40:08.602353 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:08.602385 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:08.602531 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:08.603170 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:08.603176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:08.603199 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:08.603354 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.603366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.604882 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.604886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.605925 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.605934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.607078 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.607082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.607201 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.607239 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 09:40:08.607373 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 09:40:08.607508 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":48,"thread_start_us":111,"threads_started":1}
I20260812 09:40:08.607867 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.601288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.3.147.1:36265)
0812 09:40:08.601597 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.601615 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.602020 (+ 405us) server_negotiation.cc:407] Connection header received
0812 09:40:08.602864 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.602869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.603024 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.603166 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:08.603653 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.604842 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.606423 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.606977 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.607110 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.607261 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.607580 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.607680 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":109,"threads_started":1}
I20260812 09:40:08.608311 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.601177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36265 (local address 127.0.0.1:54332)
0812 09:40:08.601875 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:08.601927 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:08.602665 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:08.603357 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:08.603360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:08.603381 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:08.603543 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.603551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.604996 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.605002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.606301 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.606310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.607328 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.607333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.607424 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.607444 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 09:40:08.608063 (+ 619us) client_negotiation.cc:240] Negotiation successful
0812 09:40:08.608174 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":62,"thread_start_us":86,"threads_started":1}
I20260812 09:40:08.608536 4845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.602052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.3.147.2:41461)
0812 09:40:08.602577 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.602589 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.602613 (+ 24us) server_negotiation.cc:407] Connection header received
0812 09:40:08.602710 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.602717 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.602993 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.603204 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:08.603500 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.604748 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.606057 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.606936 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.607045 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.607241 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.607422 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.607527 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":153,"threads_started":1}
I20260812 09:40:08.609143 4844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.602045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:43219 (local address 127.0.0.1:52778)
0812 09:40:08.602784 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:08.602808 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:08.602921 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:08.603411 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:08.603413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:08.603425 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:08.603549 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.603554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.606239 (+ 2685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.606243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.607377 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.607388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.608727 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.608730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.608825 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.608842 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 09:40:08.608930 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 09:40:08.609004 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":45,"thread_start_us":102,"threads_started":1}
I20260812 09:40:08.609402 4846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.602305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.3.147.3:43219)
0812 09:40:08.602686 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.602697 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.602908 (+ 211us) server_negotiation.cc:407] Connection header received
0812 09:40:08.603034 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.603038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.603273 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.603449 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:08.604952 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.606099 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.607521 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.608374 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.608480 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.608654 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.608976 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.609065 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":162,"threads_started":1}
I20260812 09:40:08.611505 4442 tablet_service.cc:1467] Processing CreateTablet for tablet 6a7c58de4227403596e48e137cd564b6 (DEFAULT_TABLE table=test-table [id=25f53198cce9433984dbcbc222c0dd43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:08.611778 4755 tablet_service.cc:1467] Processing CreateTablet for tablet 6a7c58de4227403596e48e137cd564b6 (DEFAULT_TABLE table=test-table [id=25f53198cce9433984dbcbc222c0dd43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:08.612941 4755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a7c58de4227403596e48e137cd564b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:08.612965 4442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a7c58de4227403596e48e137cd564b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:08.613952 4598 tablet_service.cc:1467] Processing CreateTablet for tablet 6a7c58de4227403596e48e137cd564b6 (DEFAULT_TABLE table=test-table [id=25f53198cce9433984dbcbc222c0dd43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:08.615088 4598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a7c58de4227403596e48e137cd564b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:08.620942 4847 tablet_bootstrap.cc:492] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267: Bootstrap starting.
I20260812 09:40:08.622409 4848 tablet_bootstrap.cc:492] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601: Bootstrap starting.
I20260812 09:40:08.624182 4849 tablet_bootstrap.cc:492] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3: Bootstrap starting.
I20260812 09:40:08.625244 4847 tablet_bootstrap.cc:654] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:08.625722 4848 tablet_bootstrap.cc:654] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:08.627321 4849 tablet_bootstrap.cc:654] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:08.627489 4848 log.cc:824] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:08.627489 4847 log.cc:824] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:08.628801 4849 log.cc:824] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:08.629961 4848 tablet_bootstrap.cc:492] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601: No bootstrap required, opened a new log
I20260812 09:40:08.630163 4848 ts_tablet_manager.cc:1397] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260812 09:40:08.630769 4849 tablet_bootstrap.cc:492] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3: No bootstrap required, opened a new log
I20260812 09:40:08.630978 4849 ts_tablet_manager.cc:1397] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260812 09:40:08.632726 4847 tablet_bootstrap.cc:492] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267: No bootstrap required, opened a new log
I20260812 09:40:08.633020 4847 ts_tablet_manager.cc:1397] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260812 09:40:08.635318 4848 raft_consensus.cc:348] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.635560 4848 raft_consensus.cc:374] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:08.635747 4848 raft_consensus.cc:729] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6cad435801a4cd197ef71a2d137e601, State: Initialized, Role: FOLLOWER
I20260812 09:40:08.636335 4848 consensus_queue.cc:260] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.637121 4518 heartbeater.cc:503] Master 127.3.147.62:43907 was elected leader, sending a full tablet report...
I20260812 09:40:08.637482 4848 ts_tablet_manager.cc:1428] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 09:40:08.638190 4849 raft_consensus.cc:348] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.638581 4849 raft_consensus.cc:374] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:08.638682 4849 raft_consensus.cc:729] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a0971c477146948cdf570f70a18fe3, State: Initialized, Role: FOLLOWER
I20260812 09:40:08.639326 4849 consensus_queue.cc:260] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.639648 4847 raft_consensus.cc:348] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.640030 4847 raft_consensus.cc:374] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:08.640159 4847 raft_consensus.cc:729] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a6a2f3b00d472daca48bcf1611f267, State: Initialized, Role: FOLLOWER
I20260812 09:40:08.640292 4675 heartbeater.cc:503] Master 127.3.147.62:43907 was elected leader, sending a full tablet report...
I20260812 09:40:08.640615 4847 consensus_queue.cc:260] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.641543 4829 heartbeater.cc:503] Master 127.3.147.62:43907 was elected leader, sending a full tablet report...
I20260812 09:40:08.641646 4847 ts_tablet_manager.cc:1428] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260812 09:40:08.643875 4849 ts_tablet_manager.cc:1428] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3: Time spent starting tablet: real 0.013s user 0.009s sys 0.001s
I20260812 09:40:08.725868 4854 raft_consensus.cc:482] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:08.726109 4854 raft_consensus.cc:504] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.727321 4854 leader_election.cc:288] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6cad435801a4cd197ef71a2d137e601 (127.3.147.1:36265), 72a6a2f3b00d472daca48bcf1611f267 (127.3.147.3:43219)
I20260812 09:40:08.731730 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.727634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36265 (local address 127.3.147.2:42775)
0812 09:40:08.727806 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:08.727823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:08.727927 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:08.728515 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:08.728518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:08.728540 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:08.728699 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.728705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.729577 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.729580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.730726 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.730739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.731357 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.731363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.731484 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.731514 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 09:40:08.731572 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 09:40:08.731627 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 09:40:08.731848 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.728027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:42775 (local address 127.3.147.1:36265)
0812 09:40:08.728220 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.728236 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.728257 (+ 21us) server_negotiation.cc:407] Connection header received
0812 09:40:08.728330 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.728333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.728388 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.728472 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:08.728835 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.729465 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.730868 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.731243 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.731329 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.731381 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.731634 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.731711 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 09:40:08.732800 4846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.727804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:51321 (local address 127.3.147.3:43219)
0812 09:40:08.727981 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.727988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.728683 (+ 695us) server_negotiation.cc:407] Connection header received
0812 09:40:08.728800 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.728803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.728855 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.728934 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:08.729405 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.729945 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.731472 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.732095 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.732233 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.732280 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.732650 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.732708 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 09:40:08.732825 4466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a7c58de4227403596e48e137cd564b6" candidate_uuid: "64a0971c477146948cdf570f70a18fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6cad435801a4cd197ef71a2d137e601" is_pre_election: true
I20260812 09:40:08.732934 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.727666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:43219 (local address 127.3.147.2:51321)
0812 09:40:08.728528 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:08.728541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:08.728691 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:08.728954 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:08.728957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:08.728972 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:08.729267 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.729277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.730061 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.730064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.731330 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:08.731339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.732413 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.732417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:08.732508 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.732531 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:40:08.732640 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 09:40:08.732719 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":595,"threads_started":1}
I20260812 09:40:08.733280 4466 raft_consensus.cc:2454] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64a0971c477146948cdf570f70a18fe3 in term 0.
I20260812 09:40:08.733760 4778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a7c58de4227403596e48e137cd564b6" candidate_uuid: "64a0971c477146948cdf570f70a18fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a6a2f3b00d472daca48bcf1611f267" is_pre_election: true
I20260812 09:40:08.733989 4543 leader_election.cc:302] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64a0971c477146948cdf570f70a18fe3, c6cad435801a4cd197ef71a2d137e601; no voters:
I20260812 09:40:08.734102 4778 raft_consensus.cc:2454] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64a0971c477146948cdf570f70a18fe3 in term 0.
I20260812 09:40:08.734288 4854 raft_consensus.cc:2781] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:08.734387 4854 raft_consensus.cc:482] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:08.734447 4854 raft_consensus.cc:3037] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:08.735611 4854 raft_consensus.cc:504] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.735935 4854 leader_election.cc:288] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [CANDIDATE]: Term 1 election: Requested vote from peers c6cad435801a4cd197ef71a2d137e601 (127.3.147.1:36265), 72a6a2f3b00d472daca48bcf1611f267 (127.3.147.3:43219)
I20260812 09:40:08.736306 4466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a7c58de4227403596e48e137cd564b6" candidate_uuid: "64a0971c477146948cdf570f70a18fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6cad435801a4cd197ef71a2d137e601"
I20260812 09:40:08.736405 4778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a7c58de4227403596e48e137cd564b6" candidate_uuid: "64a0971c477146948cdf570f70a18fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a6a2f3b00d472daca48bcf1611f267"
I20260812 09:40:08.736423 4466 raft_consensus.cc:3037] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:08.736534 4778 raft_consensus.cc:3037] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:08.737462 4466 raft_consensus.cc:2454] T 6a7c58de4227403596e48e137cd564b6 P c6cad435801a4cd197ef71a2d137e601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64a0971c477146948cdf570f70a18fe3 in term 1.
I20260812 09:40:08.737782 4543 leader_election.cc:302] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64a0971c477146948cdf570f70a18fe3, c6cad435801a4cd197ef71a2d137e601; no voters:
I20260812 09:40:08.737967 4854 raft_consensus.cc:2781] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:08.738296 4854 raft_consensus.cc:686] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [term 1 LEADER]: Becoming Leader. State: Replica: 64a0971c477146948cdf570f70a18fe3, State: Running, Role: LEADER
I20260812 09:40:08.738548 4778 raft_consensus.cc:2454] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64a0971c477146948cdf570f70a18fe3 in term 1.
I20260812 09:40:08.738673 4854 consensus_queue.cc:237] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } }
I20260812 09:40:08.742274 4297 catalog_manager.cc:5495] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64a0971c477146948cdf570f70a18fe3 (127.3.147.2). New cstate: current_term: 1 leader_uuid: "64a0971c477146948cdf570f70a18fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6cad435801a4cd197ef71a2d137e601" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64a0971c477146948cdf570f70a18fe3" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72a6a2f3b00d472daca48bcf1611f267" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 43219 } health_report { overall_health: UNKNOWN } } }
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:08.773490 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.756365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.3.147.1:36265)
0812 09:40:08.756495 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.756500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.756757 (+ 257us) server_negotiation.cc:407] Connection header received
0812 09:40:08.756927 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.756930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.756975 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.757059 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:08.757745 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.758246 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.759069 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.759291 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.759401 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.769377 (+ 9976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:08.769391 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:08.769407 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:08.769432 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:08.771366 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.771855 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.771860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.771862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.771947 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.772331 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.772339 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.772344 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.772729 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:08.772941 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.773253 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.773364 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260812 09:40:08.796567 4830 tablet.cc:2369] T 6a7c58de4227403596e48e137cd564b6 P 72a6a2f3b00d472daca48bcf1611f267: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:08.803314 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.794850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.3.147.1:36265)
0812 09:40:08.795041 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.795045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.795109 (+ 64us) server_negotiation.cc:407] Connection header received
0812 09:40:08.795252 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.795255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.795304 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.795376 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:08.795990 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.796561 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.797460 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.797681 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.797792 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.800105 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:08.800109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:08.800111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:08.800178 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:08.801506 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.801982 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.801986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.801987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.802065 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.802354 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.802357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.802359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.802527 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:08.802720 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.803007 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.803134 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260812 09:40:08.815173 4678 tablet.cc:2369] T 6a7c58de4227403596e48e137cd564b6 P 64a0971c477146948cdf570f70a18fe3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:08.832805 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:08.823835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.3.147.1:36265)
0812 09:40:08.824008 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:08.824013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:08.824204 (+ 191us) server_negotiation.cc:407] Connection header received
0812 09:40:08.824371 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:08.824375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:08.824428 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:08.824517 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:08.825230 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.825742 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.826680 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:08.826903 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:08.827015 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:08.829729 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:08.829732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:08.829735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:08.829758 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:08.831094 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.831521 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.831525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.831527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.831611 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:08.831902 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:08.831905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:08.831907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:08.832073 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:08.832249 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:08.832519 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 09:40:08.832640 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 09:40:08.834399 4419 tablet_service.cc:2908] Rejecting scan request for tablet 6a7c58de4227403596e48e137cd564b6: Uninitialized: clean time has not yet been initialized
I20260812 09:40:08.837061 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 4368
I20260812 09:40:08.847054 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 4523
I20260812 09:40:08.855403 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 4682
I20260812 09:40:08.865921 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 4262
2026-08-12T09:40:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2464 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4874](info): set up 2 sockets
Aug 12 09:40:08 dist-test-slave-cbbz krb5kdc[4874](info): commencing operation
krb5kdc: starting...
W20260812 09:40:09.983616 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:10 dist-test-slave-cbbz krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:10.106807 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:34813
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:34813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:10.222483 4887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:10.222805 4887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:10.222893 4887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:10.226948 4887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:10.227030 4887 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 09:40:10.227064 4887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:10.227108 4887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:10.227149 4887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:10.227185 4887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:10.232503 4887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:34813
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:34813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:10.233577 4887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:10.234627 4887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:10.243219 4893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:10.243435 4896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:10.243425 4894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:10.243425 4887 server_base.cc:1034] running on GCE node
I20260812 09:40:10.244278 4887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:10.245289 4887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:10.246456 4887 hybrid_clock.cc:648] HybridClock initialized: now 1786527610246431 us; error 37 us; skew 500 ppm
Aug 12 09:40:10 dist-test-slave-cbbz krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:10.249230 4887 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:10.250048 4887 webserver.cc:460] Webserver started at http://127.3.147.62:44713/ using document root <none> and password file <none>
I20260812 09:40:10.250612 4887 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:10.250705 4887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:10.250933 4887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:10.253036 4887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "47381f0454f0446b92b404c26876e9a2"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "8aa03ce9b50e4babc7b22e62198d507f"
server_key_iv: "72fa56570ca784d2c1fd29d42c105fa2"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.253600 4887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "47381f0454f0446b92b404c26876e9a2"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "8aa03ce9b50e4babc7b22e62198d507f"
server_key_iv: "72fa56570ca784d2c1fd29d42c105fa2"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.257185 4887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:40:10.259514 4904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:10.260743 4887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:40:10.260879 4887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "47381f0454f0446b92b404c26876e9a2"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "8aa03ce9b50e4babc7b22e62198d507f"
server_key_iv: "72fa56570ca784d2c1fd29d42c105fa2"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.260998 4887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:10.277525 4887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:10.280364 4887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:10.280560 4887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:10.288760 4887 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:34813
I20260812 09:40:10.288775 4978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:34813 every 8 connection(s)
I20260812 09:40:10.289687 4887 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:10.292500 4979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:10.293421 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 4887
I20260812 09:40:10.293540 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:10.293833 3660 external_mini_cluster.cc:1428] Setting key a08a16c39f246181ed98044833a77a55
I20260812 09:40:10.298641 4979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2: Bootstrap starting.
I20260812 09:40:10.301054 4979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:10.301833 4979 log.cc:824] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:40:10 dist-test-slave-cbbz krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:10.303813 4979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2: No bootstrap required, opened a new log
I20260812 09:40:10.306846 4979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47381f0454f0446b92b404c26876e9a2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34813 } }
I20260812 09:40:10.307083 4979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:10.307159 4979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47381f0454f0446b92b404c26876e9a2, State: Initialized, Role: FOLLOWER
I20260812 09:40:10.307662 4979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47381f0454f0446b92b404c26876e9a2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34813 } }
I20260812 09:40:10.307842 4979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:10.307909 4979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:10.308001 4979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:10.308732 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.295260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.3.147.62:34813)
0812 09:40:10.295705 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:10.295720 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:10.295753 (+ 33us) server_negotiation.cc:407] Connection header received
0812 09:40:10.296460 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:10.296478 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:10.296789 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:10.297141 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:10.297901 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.298933 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.299765 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.300295 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.300484 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.303053 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:10.303058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:10.303070 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:10.303101 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:10.305638 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.306084 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.306090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.306094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.306201 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.306490 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.306493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.306495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.306867 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:10.307064 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:10.307392 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 09:40:10.307710 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":125,"threads_started":1}
I20260812 09:40:10.309109 4979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47381f0454f0446b92b404c26876e9a2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34813 } }
I20260812 09:40:10.309448 4979 leader_election.cc:302] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47381f0454f0446b92b404c26876e9a2; no voters:
I20260812 09:40:10.309788 4979 leader_election.cc:288] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:10.309896 4984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:10.310146 4984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [term 1 LEADER]: Becoming Leader. State: Replica: 47381f0454f0446b92b404c26876e9a2, State: Running, Role: LEADER
I20260812 09:40:10.310487 4984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47381f0454f0446b92b404c26876e9a2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34813 } }
I20260812 09:40:10.311203 4979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:10.312333 4986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47381f0454f0446b92b404c26876e9a2. Latest consensus state: current_term: 1 leader_uuid: "47381f0454f0446b92b404c26876e9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47381f0454f0446b92b404c26876e9a2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34813 } } }
I20260812 09:40:10.312454 4986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:10.312697 4985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47381f0454f0446b92b404c26876e9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47381f0454f0446b92b404c26876e9a2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34813 } } }
I20260812 09:40:10.312789 4985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:10.313755 4994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:10.316861 4994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:10.322209 4994 catalog_manager.cc:1347] Generated new cluster ID: 6876a3f23ba145848b4948861259a878
I20260812 09:40:10.322284 4994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:10.334658 4994 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:10.335547 4994 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:10.344197 4994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2: Generated new TSK 0
I20260812 09:40:10.344795 4994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:10.375552 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34813
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:10.487607 5008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:10.487854 5008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:10.487926 5008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:10.492040 5008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:10.492102 5008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:10.492200 5008 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:10.492264 5008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:10.497503 5008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:10.498595 5008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:10.499637 5008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:10.509399 5013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:10.509382 5016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:10.509366 5014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:10.509869 5008 server_base.cc:1034] running on GCE node
I20260812 09:40:10.510324 5008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:10.511014 5008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:10.512223 5008 hybrid_clock.cc:648] HybridClock initialized: now 1786527610512194 us; error 60 us; skew 500 ppm
Aug 12 09:40:10 dist-test-slave-cbbz krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:10.515159 5008 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:10.516176 5008 webserver.cc:460] Webserver started at http://127.3.147.1:33763/ using document root <none> and password file <none>
I20260812 09:40:10.516785 5008 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:10.516888 5008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:10.517132 5008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:10.519258 5008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "0b18f033a53642baa8049d6ceca7998f"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "3f37fd1f4267cadfe1f833f4acb62caa"
server_key_iv: "e08777694baf1b87836b7539c74d5bcd"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.519865 5008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "0b18f033a53642baa8049d6ceca7998f"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "3f37fd1f4267cadfe1f833f4acb62caa"
server_key_iv: "e08777694baf1b87836b7539c74d5bcd"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.523906 5008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 09:40:10.526590 5023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:10.527828 5008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:10.527983 5008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "0b18f033a53642baa8049d6ceca7998f"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "3f37fd1f4267cadfe1f833f4acb62caa"
server_key_iv: "e08777694baf1b87836b7539c74d5bcd"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.528108 5008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:10.541478 5008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:10.544765 5008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:10.544986 5008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:10.546522 5008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:10.546600 5008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:10.546676 5008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:10.546727 5008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:10 dist-test-slave-cbbz krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:10.558552 5008 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:36681
I20260812 09:40:10.558586 5157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:36681 every 8 connection(s)
I20260812 09:40:10.559513 5008 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:10.562583 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 5008
I20260812 09:40:10.562709 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:10.562922 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.547660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:39403 (local address 127.3.147.62:34813)
0812 09:40:10.547838 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:10.547844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:10.548763 (+ 919us) server_negotiation.cc:407] Connection header received
0812 09:40:10.550267 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:10.550271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:10.550338 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:10.550459 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:10.552109 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.553063 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.554885 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.555321 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.555432 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.558740 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:10.558748 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:10.558751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:10.558788 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:10.560686 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.561271 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.561275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.561277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.561371 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.561711 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.561714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.561716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.561947 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:10.562058 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:10.562592 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 09:40:10.562748 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 09:40:10.563030 3660 external_mini_cluster.cc:1428] Setting key 151dd735684de0f5cbd219de869c0680
I20260812 09:40:10.563777 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.548037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34813 (local address 127.3.147.1:39403)
0812 09:40:10.548571 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:10.548625 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:10.549930 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:10.550639 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:10.550650 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:10.551123 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:10.551933 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:10.551956 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.553219 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.553223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:10.554720 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:10.554733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.555489 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.555492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:10.555694 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.556440 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:10.556464 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:10.558531 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:10.560837 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:10.560842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:10.560854 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:10.561164 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:10.561468 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:10.561470 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:10.561472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:10.561618 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:10.562068 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:10.562074 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:10.562331 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 09:40:10.562500 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 09:40:10.562828 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":155,"threads_started":1}
I20260812 09:40:10.565438 5158 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34813
I20260812 09:40:10.565659 5158 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:10.566327 5158 heartbeater.cc:511] Master 127.3.147.62:34813 requested a full tablet report, sending...
I20260812 09:40:10.568228 4925 ts_manager.cc:194] Registered new tserver with Master: 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681)
I20260812 09:40:10.569998 4925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:39403
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:10.576061 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.567923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34813 (local address 127.3.147.1:43395)
0812 09:40:10.568324 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:10.568336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:10.568456 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:10.568703 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:10.568706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:10.568925 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:10.569023 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:10.569028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.569890 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.569893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:10.570532 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:10.570541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.571130 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.571134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:10.571230 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.571839 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:10.571860 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:10.572401 (+ 541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:10.574339 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:10.574345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:10.574351 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:10.574621 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:10.574924 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:10.574928 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:10.574930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:10.575035 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:10.575569 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:10.575574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:10.575690 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 09:40:10.575810 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 09:40:10.575924 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":108,"threads_started":1}
I20260812 09:40:10.576077 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.568021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:43395 (local address 127.3.147.62:34813)
0812 09:40:10.568182 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:10.568186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:10.568412 (+ 226us) server_negotiation.cc:407] Connection header received
0812 09:40:10.568549 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:10.568552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:10.568593 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:10.568684 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:10.569142 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.569754 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.570664 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.571010 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.571123 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.572591 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:10.572594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:10.572596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:10.572619 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:10.574211 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.574740 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.574742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.574743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.574825 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.575169 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.575172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.575174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.575447 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:10.575547 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:10.575869 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 09:40:10.575965 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:10.596078 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34813
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:10.713994 5169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:10.714252 5169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:10.714299 5169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:10.718254 5169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:10.718309 5169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:10.718346 5169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:10.718379 5169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:10.723564 5169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:10.724682 5169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:10.725678 5169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:10.735693 5174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:10.735698 5177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:10.735680 5175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:10.735739 5169 server_base.cc:1034] running on GCE node
I20260812 09:40:10.736353 5169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:10.736982 5169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:10.738183 5169 hybrid_clock.cc:648] HybridClock initialized: now 1786527610738171 us; error 29 us; skew 500 ppm
Aug 12 09:40:10 dist-test-slave-cbbz krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:10.741044 5169 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:10.741852 5169 webserver.cc:460] Webserver started at http://127.3.147.2:35453/ using document root <none> and password file <none>
I20260812 09:40:10.742432 5169 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:10.742494 5169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:10.742667 5169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:10.744737 5169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "afabd2bbf0b64c14a0517af4c7547efd"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "6e5d594abf07ca16ab5af2a8da89a913"
server_key_iv: "085ed46e25760e6471ea9dca68106f09"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.745256 5169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "afabd2bbf0b64c14a0517af4c7547efd"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "6e5d594abf07ca16ab5af2a8da89a913"
server_key_iv: "085ed46e25760e6471ea9dca68106f09"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.748934 5169 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:10.751204 5185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:10.752470 5169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:10.752565 5169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "afabd2bbf0b64c14a0517af4c7547efd"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "6e5d594abf07ca16ab5af2a8da89a913"
server_key_iv: "085ed46e25760e6471ea9dca68106f09"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.752660 5169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:10.783139 5169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:10.786526 5169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:10.786693 5169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:10.788245 5169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:10.788316 5169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:10.788389 5169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:10.788441 5169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:10 dist-test-slave-cbbz krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:10.799942 5169 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46169
I20260812 09:40:10.800328 5321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46169 every 8 connection(s)
I20260812 09:40:10.801093 5169 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:10.803120 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 5169
I20260812 09:40:10.803261 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:10.803560 3660 external_mini_cluster.cc:1428] Setting key 44777360952de03c8170d882f0a38339
I20260812 09:40:10.803560 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.789208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:56491 (local address 127.3.147.62:34813)
0812 09:40:10.789353 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:10.789357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:10.790461 (+ 1104us) server_negotiation.cc:407] Connection header received
0812 09:40:10.791572 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:10.791576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:10.791643 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:10.791735 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:10.793525 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.794112 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.795221 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.795432 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.795560 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.799636 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:10.799641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:10.799646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:10.799680 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:10.801351 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.802151 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.802154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.802155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.802231 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.802547 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.802549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.802550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.802726 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:10.802802 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:10.803269 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 09:40:10.803395 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:40:10.804504 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.789720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34813 (local address 127.3.147.2:56491)
0812 09:40:10.790303 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:10.790355 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:10.791317 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:10.792256 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:10.792263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:10.792713 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:10.793361 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:10.793383 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.794245 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.794249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:10.795084 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:10.795102 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.795616 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.795620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:10.795856 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.796486 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:10.796504 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:10.799414 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:10.801493 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:10.801501 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:10.801515 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:10.801885 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:10.802321 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:10.802324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:10.802325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:10.802459 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:10.802811 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:10.802816 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:10.803028 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 09:40:10.803180 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 09:40:10.803491 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":151,"threads_started":1}
I20260812 09:40:10.805859 5322 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34813
I20260812 09:40:10.806002 5322 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:10.806565 5322 heartbeater.cc:511] Master 127.3.147.62:34813 requested a full tablet report, sending...
I20260812 09:40:10.807678 4925 ts_manager.cc:194] Registered new tserver with Master: afabd2bbf0b64c14a0517af4c7547efd (127.3.147.2:46169)
I20260812 09:40:10.808440 4925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:56491
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:40:10.815523 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.808431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:50747 (local address 127.3.147.62:34813)
0812 09:40:10.808556 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:10.808563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:10.808580 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:40:10.808694 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:10.808697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:10.808743 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:10.808831 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:10.809357 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.809854 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.810813 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.811042 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.811149 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.812398 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:10.812405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:10.812408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:10.812435 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:10.813893 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.814333 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.814335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.814336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.814408 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:10.814689 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:10.814693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:10.814695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:10.814894 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:10.815002 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:10.815266 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 09:40:10.815383 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 09:40:10.815542 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:10.808179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34813 (local address 127.3.147.2:50747)
0812 09:40:10.808505 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:10.808521 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:10.808608 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:10.808843 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:10.808845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:10.809060 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:10.809183 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:10.809190 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.809983 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.809987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:10.810685 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:10.810695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:10.811155 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:10.811159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:10.811354 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:10.811855 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:10.811868 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:10.812245 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:10.814006 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:10.814011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:10.814013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:10.814209 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:10.814505 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:10.814509 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:10.814511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:10.814597 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:10.815007 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:10.815011 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:10.815104 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 09:40:10.815210 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 09:40:10.815355 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":2,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:10.833197 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34813
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:10.946779 5332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:10.947068 5332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:10.947155 5332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:10.951807 5332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:10.951913 5332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:10.951980 5332 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:10.952060 5332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:10.957549 5332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:10.958696 5332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:10.959749 5332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:10.969640 5342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:10.969640 5340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:10.969842 5332 server_base.cc:1034] running on GCE node
W20260812 09:40:10.969650 5339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:10.970309 5332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:10.970954 5332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:10.972237 5332 hybrid_clock.cc:648] HybridClock initialized: now 1786527610972223 us; error 25 us; skew 500 ppm
Aug 12 09:40:10 dist-test-slave-cbbz krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:10.975029 5332 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:10.976007 5332 webserver.cc:460] Webserver started at http://127.3.147.3:40325/ using document root <none> and password file <none>
I20260812 09:40:10.976637 5332 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:10.976734 5332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:10.976974 5332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:10.979135 5332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "3d427e207f9449718f324cae6691ecac"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "0c0d934002f92a6c437c8384a7e20246"
server_key_iv: "f16d9405280acf130fc328c7e9da1fb3"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.979723 5332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "3d427e207f9449718f324cae6691ecac"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "0c0d934002f92a6c437c8384a7e20246"
server_key_iv: "f16d9405280acf130fc328c7e9da1fb3"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.983536 5332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 09:40:10.986065 5349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:10.987282 5332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:10.987417 5332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "3d427e207f9449718f324cae6691ecac"
format_stamp: "Formatted at 2026-08-12 09:40:10 on dist-test-slave-cbbz"
server_key: "0c0d934002f92a6c437c8384a7e20246"
server_key_iv: "f16d9405280acf130fc328c7e9da1fb3"
server_key_version: "encryptionkey@0"
I20260812 09:40:10.987555 5332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:11.005537 5332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:11.009009 5332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:11.009218 5332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:11.010783 5332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:11.010862 5332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:11.010929 5332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:11.010979 5332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:11 dist-test-slave-cbbz krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:11.023188 5332 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:39933
I20260812 09:40:11.023190 5486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:39933 every 8 connection(s)
I20260812 09:40:11.024350 5332 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:11.026165 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.012069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34813 (local address 127.3.147.3:55079)
0812 09:40:11.012689 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.012749 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.013940 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.014586 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.014595 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.014923 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:11.015437 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.015450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.016355 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.016359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.017327 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.017336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.017866 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.017870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.018031 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.018608 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:11.018630 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:11.021050 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:11.023402 (+ 2352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:11.023408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:11.023422 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:11.023684 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:11.024020 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:11.024023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:11.024024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:11.024202 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:11.024683 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:11.024707 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:11.024989 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.025171 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.025455 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":125,"threads_started":1}
I20260812 09:40:11.026978 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.011579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:55079 (local address 127.3.147.62:34813)
0812 09:40:11.011733 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.011738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.012864 (+ 1126us) server_negotiation.cc:407] Connection header received
0812 09:40:11.014226 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.014231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.014296 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.014405 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:11.015590 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.016233 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.017456 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.017707 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.017809 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.021230 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:11.021236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:11.021240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:11.021276 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:11.023246 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.023789 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.023793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.023795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.023893 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.024313 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.024319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.024324 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.024555 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:11.024686 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.026767 (+ 2081us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.026870 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 09:40:11.028051 5487 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34813
I20260812 09:40:11.028273 5487 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:11.028784 5487 heartbeater.cc:511] Master 127.3.147.62:34813 requested a full tablet report, sending...
I20260812 09:40:11.029808 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 5332
I20260812 09:40:11.029842 4924 ts_manager.cc:194] Registered new tserver with Master: 3d427e207f9449718f324cae6691ecac (127.3.147.3:39933)
I20260812 09:40:11.029940 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:11.030282 3660 external_mini_cluster.cc:1428] Setting key 2627b96a28d300466956a9ae8dc8286c
I20260812 09:40:11.030663 4924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:55079
I20260812 09:40:11.032569 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:40:11.037930 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.030273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34813 (local address 127.3.147.3:55851)
0812 09:40:11.030652 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.030665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.030789 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.031043 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.031049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.031265 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:11.031405 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.031413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.032408 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.032411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.033290 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.033299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.033923 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.033925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.033993 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.034486 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:11.034497 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:11.034792 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:11.036554 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:11.036556 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:11.036558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:11.036743 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:11.037037 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:11.037039 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:11.037040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:11.037111 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:11.037545 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:11.037547 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:11.037617 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.037705 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.037804 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260812 09:40:11.038110 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.030365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:55851 (local address 127.3.147.62:34813)
0812 09:40:11.030476 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.030480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.030750 (+ 270us) server_negotiation.cc:407] Connection header received
0812 09:40:11.030866 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.030869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.030916 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.030993 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:11.031529 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.032274 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.033417 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.033804 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.033928 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.034937 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:11.034944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:11.034950 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:11.034983 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:11.036440 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.036840 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.036846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.036851 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.036940 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.037204 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.037210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.037215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.037448 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:11.037557 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.037804 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.037937 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 09:40:11 dist-test-slave-cbbz krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527611, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 09:40:11 dist-test-slave-cbbz krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527611, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:11.059518 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.051667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58204 (local address 127.3.147.62:34813)
0812 09:40:11.051794 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.051797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.051807 (+ 10us) server_negotiation.cc:407] Connection header received
0812 09:40:11.051837 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.051840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.051873 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.051957 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:11.052694 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.053201 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.054023 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.054240 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.054347 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.056592 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:11.056596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:11.056599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:11.056623 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:11.057904 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.058329 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.058333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.058335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.058417 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.058698 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.058702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.058704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.058882 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:11.059017 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.059254 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.059357 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:40:11.062170 4925 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:11.065280 4925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:11.079403 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.074376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56164 (local address 127.3.147.2:46169)
0812 09:40:11.074842 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.074854 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.074875 (+ 21us) server_negotiation.cc:407] Connection header received
0812 09:40:11.074944 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.074949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.075146 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.075550 (+ 404us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:11.075661 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.076496 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.077907 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.078740 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.078878 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.079058 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.079166 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.079258 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":125,"threads_started":1}
I20260812 09:40:11.079406 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.074053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46169 (local address 127.0.0.1:56164)
0812 09:40:11.074413 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.074427 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.074513 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.075293 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.075298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.075311 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:11.075422 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.075430 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.076615 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.076619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.077785 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.077794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.078882 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.078886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.078987 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.079027 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.079139 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.079253 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":34,"thread_start_us":99,"threads_started":1}
I20260812 09:40:11.079734 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.073396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35230 (local address 127.3.147.1:36681)
0812 09:40:11.073819 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.073825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.073849 (+ 24us) server_negotiation.cc:407] Connection header received
0812 09:40:11.074507 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.074513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.074728 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.074874 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:11.075165 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.076381 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.078097 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.078858 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.078947 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.079066 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.079483 (+ 417us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.079583 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":87,"threads_started":1}
I20260812 09:40:11.080032 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.073256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36681 (local address 127.0.0.1:35230)
0812 09:40:11.073685 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.073734 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.073855 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.074876 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.074882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.074909 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:11.075028 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.075034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.076499 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.076503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.077971 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.077982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.079124 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.079130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.079222 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.079241 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.079309 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.079366 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":27,"thread_start_us":160,"threads_started":1}
I20260812 09:40:11.080771 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.073491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:39933 (local address 127.0.0.1:58846)
0812 09:40:11.073989 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.074002 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.074081 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.075184 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.075186 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.075206 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:11.075319 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.075326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.076763 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.076767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.077972 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.077982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.080077 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.080080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.080458 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.080482 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.080555 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.080636 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":40,"thread_start_us":92,"threads_started":1}
I20260812 09:40:11.081209 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.074144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.3.147.3:39933)
0812 09:40:11.074766 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.074782 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.074806 (+ 24us) server_negotiation.cc:407] Connection header received
0812 09:40:11.074878 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.074882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.075051 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.075214 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:11.075438 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.076652 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.079140 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.079931 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.080056 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.080249 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.080941 (+ 692us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.081067 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":136,"threads_started":1}
I20260812 09:40:11.082592 5243 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa9f48f789948a2967703c736e6b4da (DEFAULT_TABLE table=test-table [id=c7255d761279412e93f7a1a8a7332fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:11.082916 5081 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa9f48f789948a2967703c736e6b4da (DEFAULT_TABLE table=test-table [id=c7255d761279412e93f7a1a8a7332fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:11.083580 5243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa9f48f789948a2967703c736e6b4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.083904 5081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa9f48f789948a2967703c736e6b4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.083793 5408 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa9f48f789948a2967703c736e6b4da (DEFAULT_TABLE table=test-table [id=c7255d761279412e93f7a1a8a7332fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:11.084753 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa9f48f789948a2967703c736e6b4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.090484 5515 tablet_bootstrap.cc:492] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: Bootstrap starting.
I20260812 09:40:11.092197 5517 tablet_bootstrap.cc:492] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: Bootstrap starting.
I20260812 09:40:11.092491 5516 tablet_bootstrap.cc:492] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: Bootstrap starting.
I20260812 09:40:11.093433 5515 tablet_bootstrap.cc:654] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.094285 5517 tablet_bootstrap.cc:654] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.094310 5515 log.cc:824] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:11.094575 5516 tablet_bootstrap.cc:654] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.095139 5517 log.cc:824] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:11.095309 5516 log.cc:824] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:11.096446 5515 tablet_bootstrap.cc:492] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: No bootstrap required, opened a new log
I20260812 09:40:11.096653 5515 ts_tablet_manager.cc:1397] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:40:11.097055 5516 tablet_bootstrap.cc:492] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: No bootstrap required, opened a new log
I20260812 09:40:11.097186 5517 tablet_bootstrap.cc:492] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: No bootstrap required, opened a new log
I20260812 09:40:11.097277 5516 ts_tablet_manager.cc:1397] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 09:40:11.097363 5517 ts_tablet_manager.cc:1397] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:40:11.100747 5516 raft_consensus.cc:348] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.100684 5515 raft_consensus.cc:348] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.100965 5516 raft_consensus.cc:374] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.100966 5515 raft_consensus.cc:374] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.101029 5515 raft_consensus.cc:729] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afabd2bbf0b64c14a0517af4c7547efd, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.101047 5516 raft_consensus.cc:729] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b18f033a53642baa8049d6ceca7998f, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.101140 5517 raft_consensus.cc:348] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.101428 5517 raft_consensus.cc:374] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.101487 5517 raft_consensus.cc:729] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d427e207f9449718f324cae6691ecac, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.101475 5515 consensus_queue.cc:260] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.101475 5516 consensus_queue.cc:260] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.101923 5517 consensus_queue.cc:260] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.102089 5158 heartbeater.cc:503] Master 127.3.147.62:34813 was elected leader, sending a full tablet report...
I20260812 09:40:11.102427 5516 ts_tablet_manager.cc:1428] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 09:40:11.102461 5515 ts_tablet_manager.cc:1428] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260812 09:40:11.102835 5517 ts_tablet_manager.cc:1428] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 09:40:11.103102 5322 heartbeater.cc:503] Master 127.3.147.62:34813 was elected leader, sending a full tablet report...
I20260812 09:40:11.103116 5487 heartbeater.cc:503] Master 127.3.147.62:34813 was elected leader, sending a full tablet report...
I20260812 09:40:11.127723 5522 raft_consensus.cc:482] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:11.127961 5522 raft_consensus.cc:504] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.129135 5522 leader_election.cc:288] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681), 3d427e207f9449718f324cae6691ecac (127.3.147.3:39933)
I20260812 09:40:11.133702 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.129397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36681 (local address 127.3.147.2:42241)
0812 09:40:11.129526 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.129542 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.129629 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.129891 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.129894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.129914 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:11.130064 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.130075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.131106 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.131111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.132505 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.132517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.133300 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.133304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.133397 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.133418 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.133490 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.133549 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 09:40:11.133749 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.129500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:42241 (local address 127.3.147.1:36681)
0812 09:40:11.129634 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.129639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.129657 (+ 18us) server_negotiation.cc:407] Connection header received
0812 09:40:11.129717 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.129722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.129775 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.129858 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:11.130197 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.130983 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.132627 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.133155 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.133266 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.133330 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.133525 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.133600 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 09:40:11.134235 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.129560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:59213 (local address 127.3.147.3:39933)
0812 09:40:11.129703 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.129710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.130011 (+ 301us) server_negotiation.cc:407] Connection header received
0812 09:40:11.130098 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.130101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.130189 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.130270 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:11.130509 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.131211 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.133042 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.133606 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.133712 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.133771 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.134046 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.134109 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:40:11.134310 5524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.129417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:39933 (local address 127.3.147.2:59213)
0812 09:40:11.129911 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.129924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.130016 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.130288 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.130290 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.130306 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:11.130401 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.130404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.131301 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.131332 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.132931 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.132939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.133764 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.133769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.133909 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.133943 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.134019 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.134106 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":107,"thread_start_us":158,"threads_started":1}
I20260812 09:40:11.134765 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9f48f789948a2967703c736e6b4da" candidate_uuid: "afabd2bbf0b64c14a0517af4c7547efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18f033a53642baa8049d6ceca7998f" is_pre_election: true
I20260812 09:40:11.135076 5106 raft_consensus.cc:2454] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afabd2bbf0b64c14a0517af4c7547efd in term 0.
I20260812 09:40:11.135072 5433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9f48f789948a2967703c736e6b4da" candidate_uuid: "afabd2bbf0b64c14a0517af4c7547efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d427e207f9449718f324cae6691ecac" is_pre_election: true
I20260812 09:40:11.135347 5433 raft_consensus.cc:2454] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afabd2bbf0b64c14a0517af4c7547efd in term 0.
I20260812 09:40:11.135636 5187 leader_election.cc:302] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b18f033a53642baa8049d6ceca7998f, afabd2bbf0b64c14a0517af4c7547efd; no voters:
I20260812 09:40:11.135916 5522 raft_consensus.cc:2781] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:11.136005 5522 raft_consensus.cc:482] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:11.136049 5522 raft_consensus.cc:3037] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.137230 5522 raft_consensus.cc:504] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.137638 5522 leader_election.cc:288] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [CANDIDATE]: Term 1 election: Requested vote from peers 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681), 3d427e207f9449718f324cae6691ecac (127.3.147.3:39933)
I20260812 09:40:11.137960 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9f48f789948a2967703c736e6b4da" candidate_uuid: "afabd2bbf0b64c14a0517af4c7547efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18f033a53642baa8049d6ceca7998f"
I20260812 09:40:11.138091 5106 raft_consensus.cc:3037] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.139127 5106 raft_consensus.cc:2454] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afabd2bbf0b64c14a0517af4c7547efd in term 1.
I20260812 09:40:11.139429 5433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9f48f789948a2967703c736e6b4da" candidate_uuid: "afabd2bbf0b64c14a0517af4c7547efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d427e207f9449718f324cae6691ecac"
I20260812 09:40:11.139469 5187 leader_election.cc:302] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b18f033a53642baa8049d6ceca7998f, afabd2bbf0b64c14a0517af4c7547efd; no voters:
I20260812 09:40:11.139580 5433 raft_consensus.cc:3037] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.139639 5522 raft_consensus.cc:2781] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:11.139858 5522 raft_consensus.cc:686] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 1 LEADER]: Becoming Leader. State: Replica: afabd2bbf0b64c14a0517af4c7547efd, State: Running, Role: LEADER
I20260812 09:40:11.140265 5522 consensus_queue.cc:237] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } }
I20260812 09:40:11.140990 5433 raft_consensus.cc:2454] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afabd2bbf0b64c14a0517af4c7547efd in term 1.
I20260812 09:40:11.143695 4921 catalog_manager.cc:5495] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd reported cstate change: term changed from 0 to 1, leader changed from <none> to afabd2bbf0b64c14a0517af4c7547efd (127.3.147.2). New cstate: current_term: 1 leader_uuid: "afabd2bbf0b64c14a0517af4c7547efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } health_report { overall_health: HEALTHY } } }
I20260812 09:40:11.169497 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.165926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56174 (local address 127.3.147.2:46169)
0812 09:40:11.166074 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.166078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.166093 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:40:11.166291 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.166294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.166351 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.166440 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:11.166672 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.167189 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.168019 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.168320 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.168404 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.168694 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:11.169123 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:11.169213 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.169335 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.169384 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 09:40:11.175712 5106 raft_consensus.cc:1261] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Refusing update from remote peer afabd2bbf0b64c14a0517af4c7547efd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:11.175966 5433 raft_consensus.cc:1261] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Refusing update from remote peer afabd2bbf0b64c14a0517af4c7547efd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:11.176663 5525 consensus_queue.cc:1035] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:11.176908 5522 consensus_queue.cc:1035] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:11.197319 4924 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:11.197594 4924 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:11.200718 4924 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2: Sending DeleteTablet for 3 replicas of tablet 5fa9f48f789948a2967703c736e6b4da
I20260812 09:40:11.201653 5081 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa9f48f789948a2967703c736e6b4da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:11 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35230
I20260812 09:40:11.201709 5408 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa9f48f789948a2967703c736e6b4da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:11 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:58846
I20260812 09:40:11.202013 5243 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa9f48f789948a2967703c736e6b4da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:11 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:56164
I20260812 09:40:11.202446 5538 tablet_replica.cc:330] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: stopping tablet replica
I20260812 09:40:11.202515 5539 tablet_replica.cc:330] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: stopping tablet replica
I20260812 09:40:11.202872 5538 raft_consensus.cc:2229] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:11.202872 5539 raft_consensus.cc:2229] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:11.202927 4924 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 09:40:11.203313 5539 raft_consensus.cc:2258] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:11.203420 5538 raft_consensus.cc:2258] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 09:40:11.203454 4924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:11.204988 5539 ts_tablet_manager.cc:1905] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:11.205454 5538 ts_tablet_manager.cc:1905] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:11.205442 5540 tablet_replica.cc:330] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: stopping tablet replica
I20260812 09:40:11.205832 5540 raft_consensus.cc:2229] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:11.206269 5540 raft_consensus.cc:2258] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:11.209272 5408 tablet_service.cc:1467] Processing CreateTablet for tablet 489d11c548f44ddfa55dd802fcfedb7b (DEFAULT_TABLE table=test-table [id=00b52235f1954672bb85cf16a51affb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:11.209272 5081 tablet_service.cc:1467] Processing CreateTablet for tablet 489d11c548f44ddfa55dd802fcfedb7b (DEFAULT_TABLE table=test-table [id=00b52235f1954672bb85cf16a51affb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:11.209532 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 489d11c548f44ddfa55dd802fcfedb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.209532 5081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 489d11c548f44ddfa55dd802fcfedb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.210902 5243 tablet_service.cc:1467] Processing CreateTablet for tablet 489d11c548f44ddfa55dd802fcfedb7b (DEFAULT_TABLE table=test-table [id=00b52235f1954672bb85cf16a51affb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:11.211171 5243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 489d11c548f44ddfa55dd802fcfedb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.211328 5539 ts_tablet_manager.cc:1918] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:11.211434 5539 log.cc:1196] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/5fa9f48f789948a2967703c736e6b4da
I20260812 09:40:11.211824 5539 ts_tablet_manager.cc:1939] T 5fa9f48f789948a2967703c736e6b4da P 3d427e207f9449718f324cae6691ecac: Deleting consensus metadata
I20260812 09:40:11.213449 5540 ts_tablet_manager.cc:1905] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:11.213455 4907 catalog_manager.cc:4841] TS 3d427e207f9449718f324cae6691ecac (127.3.147.3:39933): tablet 5fa9f48f789948a2967703c736e6b4da (table test-table [id=c7255d761279412e93f7a1a8a7332fc9]) successfully deleted
I20260812 09:40:11.213941 5517 tablet_bootstrap.cc:492] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: Bootstrap starting.
I20260812 09:40:11.215803 5517 tablet_bootstrap.cc:654] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.217006 5540 ts_tablet_manager.cc:1918] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:11.217051 5516 tablet_bootstrap.cc:492] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: Bootstrap starting.
I20260812 09:40:11.217101 5540 log.cc:1196] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/5fa9f48f789948a2967703c736e6b4da
I20260812 09:40:11.217450 5540 ts_tablet_manager.cc:1939] T 5fa9f48f789948a2967703c736e6b4da P afabd2bbf0b64c14a0517af4c7547efd: Deleting consensus metadata
I20260812 09:40:11.217906 5515 tablet_bootstrap.cc:492] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: Bootstrap starting.
I20260812 09:40:11.218609 5516 tablet_bootstrap.cc:654] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.219333 5538 ts_tablet_manager.cc:1918] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:11.219592 5538 log.cc:1196] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/5fa9f48f789948a2967703c736e6b4da
I20260812 09:40:11.219801 5515 tablet_bootstrap.cc:654] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.221410 5538 ts_tablet_manager.cc:1939] T 5fa9f48f789948a2967703c736e6b4da P 0b18f033a53642baa8049d6ceca7998f: Deleting consensus metadata
I20260812 09:40:11.221417 4906 catalog_manager.cc:4841] TS afabd2bbf0b64c14a0517af4c7547efd (127.3.147.2:46169): tablet 5fa9f48f789948a2967703c736e6b4da (table test-table [id=c7255d761279412e93f7a1a8a7332fc9]) successfully deleted
I20260812 09:40:11.221561 5516 tablet_bootstrap.cc:492] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: No bootstrap required, opened a new log
I20260812 09:40:11.221640 5516 ts_tablet_manager.cc:1397] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 09:40:11.222808 5516 raft_consensus.cc:348] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.222949 5516 raft_consensus.cc:374] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.222986 5516 raft_consensus.cc:729] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b18f033a53642baa8049d6ceca7998f, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.223061 5515 tablet_bootstrap.cc:492] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: No bootstrap required, opened a new log
I20260812 09:40:11.223147 5515 ts_tablet_manager.cc:1397] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 09:40:11.223099 5516 consensus_queue.cc:260] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.223368 5516 ts_tablet_manager.cc:1428] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:11.224270 5515 raft_consensus.cc:348] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.224416 5515 raft_consensus.cc:374] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.224457 5515 raft_consensus.cc:729] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afabd2bbf0b64c14a0517af4c7547efd, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.224618 5515 consensus_queue.cc:260] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.224929 5515 ts_tablet_manager.cc:1428] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:11.225145 4906 catalog_manager.cc:4841] TS 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681): tablet 5fa9f48f789948a2967703c736e6b4da (table test-table [id=c7255d761279412e93f7a1a8a7332fc9]) successfully deleted
I20260812 09:40:11.225698 5517 tablet_bootstrap.cc:492] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: No bootstrap required, opened a new log
I20260812 09:40:11.225811 5517 ts_tablet_manager.cc:1397] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.003s
I20260812 09:40:11.226244 5517 raft_consensus.cc:348] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.226375 5517 raft_consensus.cc:374] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.226423 5517 raft_consensus.cc:729] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d427e207f9449718f324cae6691ecac, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.226577 5517 consensus_queue.cc:260] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.226934 5517 ts_tablet_manager.cc:1428] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260812 09:40:11.275758 5489 tablet.cc:2369] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:40:11.302703 5324 tablet.cc:2369] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:40:11.311077 5159 tablet.cc:2369] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:40:11.339252 5523 raft_consensus.cc:482] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:11.339464 5523 raft_consensus.cc:504] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.340593 5523 leader_election.cc:288] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681), afabd2bbf0b64c14a0517af4c7547efd (127.3.147.2:46169)
I20260812 09:40:11.345484 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.341092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:54995 (local address 127.3.147.1:36681)
0812 09:40:11.341234 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.341241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.341329 (+ 88us) server_negotiation.cc:407] Connection header received
0812 09:40:11.341495 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.341499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.341552 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.341641 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:11.342184 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.343115 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.344490 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.344896 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.345017 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.345086 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.345303 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.345381 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 09:40:11.345631 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.341075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36681 (local address 127.3.147.3:54995)
0812 09:40:11.341226 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.341242 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.341387 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.341767 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.341771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.341816 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:11.341996 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.342011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.343277 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.343280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.344355 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.344365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.345019 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.345022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.345151 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.345181 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.345273 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.345346 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":20}
I20260812 09:40:11.346560 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "489d11c548f44ddfa55dd802fcfedb7b" candidate_uuid: "3d427e207f9449718f324cae6691ecac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18f033a53642baa8049d6ceca7998f" is_pre_election: true
I20260812 09:40:11.346740 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.341983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:36291 (local address 127.3.147.2:46169)
0812 09:40:11.342147 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.342152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.342171 (+ 19us) server_negotiation.cc:407] Connection header received
0812 09:40:11.342216 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.342218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.342255 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.342351 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:11.342808 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.343359 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.345710 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.346113 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.346246 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.346315 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.346570 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.346636 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 09:40:11.346743 5106 raft_consensus.cc:2454] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d427e207f9449718f324cae6691ecac in term 0.
I20260812 09:40:11.346856 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.341425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46169 (local address 127.3.147.3:36291)
0812 09:40:11.341759 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.341772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.341867 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.342364 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.342368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.342386 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:11.342686 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.342694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.343490 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.343494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.345505 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.345518 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.346247 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.346250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.346416 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.346443 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.346548 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.346612 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":39,"thread_start_us":115,"threads_started":1}
I20260812 09:40:11.347602 5351 leader_election.cc:302] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b18f033a53642baa8049d6ceca7998f, 3d427e207f9449718f324cae6691ecac; no voters:
I20260812 09:40:11.347893 5523 raft_consensus.cc:2781] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:11.347970 5523 raft_consensus.cc:482] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:11.348018 5523 raft_consensus.cc:3037] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.348021 5264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "489d11c548f44ddfa55dd802fcfedb7b" candidate_uuid: "3d427e207f9449718f324cae6691ecac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afabd2bbf0b64c14a0517af4c7547efd" is_pre_election: true
I20260812 09:40:11.348304 5264 raft_consensus.cc:2454] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d427e207f9449718f324cae6691ecac in term 0.
I20260812 09:40:11.349921 5523 raft_consensus.cc:504] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.350358 5523 leader_election.cc:288] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [CANDIDATE]: Term 1 election: Requested vote from peers 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681), afabd2bbf0b64c14a0517af4c7547efd (127.3.147.2:46169)
I20260812 09:40:11.350795 5264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "489d11c548f44ddfa55dd802fcfedb7b" candidate_uuid: "3d427e207f9449718f324cae6691ecac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afabd2bbf0b64c14a0517af4c7547efd"
I20260812 09:40:11.350826 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "489d11c548f44ddfa55dd802fcfedb7b" candidate_uuid: "3d427e207f9449718f324cae6691ecac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18f033a53642baa8049d6ceca7998f"
I20260812 09:40:11.350929 5264 raft_consensus.cc:3037] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.350929 5106 raft_consensus.cc:3037] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.352804 5106 raft_consensus.cc:2454] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d427e207f9449718f324cae6691ecac in term 1.
I20260812 09:40:11.352967 5264 raft_consensus.cc:2454] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d427e207f9449718f324cae6691ecac in term 1.
I20260812 09:40:11.353128 5351 leader_election.cc:302] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b18f033a53642baa8049d6ceca7998f, 3d427e207f9449718f324cae6691ecac; no voters:
I20260812 09:40:11.353289 5523 raft_consensus.cc:2781] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:11.353538 5523 raft_consensus.cc:686] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 1 LEADER]: Becoming Leader. State: Replica: 3d427e207f9449718f324cae6691ecac, State: Running, Role: LEADER
I20260812 09:40:11.353788 5523 consensus_queue.cc:237] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.356348 4924 catalog_manager.cc:5495] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d427e207f9449718f324cae6691ecac (127.3.147.3). New cstate: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } health_report { overall_health: HEALTHY } } }
Aug 12 09:40:11 dist-test-slave-cbbz krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527610, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:11.419014 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.410000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34813 (local address 127.0.0.1:58216)
0812 09:40:11.410407 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.410419 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.410618 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.410921 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.410924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.411395 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:11.411506 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.411514 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.412484 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.412488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.413140 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.413150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.413570 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.413573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.413643 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.414214 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:11.414225 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:11.415775 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:11.417364 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:11.417375 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:11.417381 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:11.417626 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:11.417917 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:11.417921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:11.417923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:11.418004 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:11.418385 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:11.418393 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:11.418498 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.418616 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.418763 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":112,"threads_started":1}
I20260812 09:40:11.419013 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.410086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58216 (local address 127.3.147.62:34813)
0812 09:40:11.410234 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.410241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.410554 (+ 313us) server_negotiation.cc:407] Connection header received
0812 09:40:11.410741 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.410747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.410805 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.410903 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:11.411674 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.412364 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.413254 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.413463 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.413568 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.415929 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:11.415933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:11.415936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:11.415967 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:11.417255 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.417731 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.417736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.417738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.417823 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.418101 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.418104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.418106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.418268 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:11.418375 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.418662 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.418827 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":44}
I20260812 09:40:11.423194 4924 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:58216:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 09:40:11.423882 4924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:11.428973 5081 tablet_service.cc:1467] Processing CreateTablet for tablet 7beb05c96af6460ebc53dfc0b3288341 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=772cb18df6994b1b996e8e2f12fb1bfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:11.429138 5243 tablet_service.cc:1467] Processing CreateTablet for tablet 7beb05c96af6460ebc53dfc0b3288341 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=772cb18df6994b1b996e8e2f12fb1bfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:11.429178 5408 tablet_service.cc:1467] Processing CreateTablet for tablet 7beb05c96af6460ebc53dfc0b3288341 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=772cb18df6994b1b996e8e2f12fb1bfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:11.429437 5081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7beb05c96af6460ebc53dfc0b3288341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.429478 5243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7beb05c96af6460ebc53dfc0b3288341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.429615 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7beb05c96af6460ebc53dfc0b3288341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:11.432750 5515 tablet_bootstrap.cc:492] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: Bootstrap starting.
I20260812 09:40:11.432775 5516 tablet_bootstrap.cc:492] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: Bootstrap starting.
I20260812 09:40:11.432941 5517 tablet_bootstrap.cc:492] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: Bootstrap starting.
I20260812 09:40:11.434001 5515 tablet_bootstrap.cc:654] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.434543 5517 tablet_bootstrap.cc:654] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.434582 5516 tablet_bootstrap.cc:654] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:11.435708 5515 tablet_bootstrap.cc:492] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: No bootstrap required, opened a new log
I20260812 09:40:11.435810 5515 ts_tablet_manager.cc:1397] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:11.436342 5515 raft_consensus.cc:348] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.436479 5515 raft_consensus.cc:374] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.436522 5515 raft_consensus.cc:729] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afabd2bbf0b64c14a0517af4c7547efd, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.436676 5515 consensus_queue.cc:260] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.436990 5515 ts_tablet_manager.cc:1428] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:11.436971 5535 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.437165 5535 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:11.436802 5516 tablet_bootstrap.cc:492] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: No bootstrap required, opened a new log
I20260812 09:40:11.437345 5516 ts_tablet_manager.cc:1397] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:40:11.437512 5517 tablet_bootstrap.cc:492] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: No bootstrap required, opened a new log
I20260812 09:40:11.437624 5517 ts_tablet_manager.cc:1397] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:40:11.438090 5516 raft_consensus.cc:348] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.438228 5516 raft_consensus.cc:374] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.438272 5516 raft_consensus.cc:729] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b18f033a53642baa8049d6ceca7998f, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.438306 5517 raft_consensus.cc:348] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.438421 5516 consensus_queue.cc:260] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.438496 5517 raft_consensus.cc:374] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:11.438542 5517 raft_consensus.cc:729] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d427e207f9449718f324cae6691ecac, State: Initialized, Role: FOLLOWER
I20260812 09:40:11.438680 5517 consensus_queue.cc:260] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.438657 5521 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.438927 5521 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:11.439102 5517 ts_tablet_manager.cc:1428] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:11.439466 5523 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.439595 5523 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:11.439692 5516 ts_tablet_manager.cc:1428] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:11.443537 5523 raft_consensus.cc:482] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:11.443642 5523 raft_consensus.cc:504] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.444085 5523 leader_election.cc:288] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681), afabd2bbf0b64c14a0517af4c7547efd (127.3.147.2:46169)
I20260812 09:40:11.444440 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7beb05c96af6460ebc53dfc0b3288341" candidate_uuid: "3d427e207f9449718f324cae6691ecac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18f033a53642baa8049d6ceca7998f" is_pre_election: true
I20260812 09:40:11.444588 5264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7beb05c96af6460ebc53dfc0b3288341" candidate_uuid: "3d427e207f9449718f324cae6691ecac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afabd2bbf0b64c14a0517af4c7547efd" is_pre_election: true
I20260812 09:40:11.444643 5106 raft_consensus.cc:2454] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d427e207f9449718f324cae6691ecac in term 0.
I20260812 09:40:11.444742 5264 raft_consensus.cc:2454] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d427e207f9449718f324cae6691ecac in term 0.
I20260812 09:40:11.445011 5351 leader_election.cc:302] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b18f033a53642baa8049d6ceca7998f, 3d427e207f9449718f324cae6691ecac; no voters:
I20260812 09:40:11.445312 5523 raft_consensus.cc:2781] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:11.445381 5523 raft_consensus.cc:482] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:11.445433 5523 raft_consensus.cc:3037] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.446480 5523 raft_consensus.cc:504] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.446787 5523 leader_election.cc:288] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [CANDIDATE]: Term 1 election: Requested vote from peers 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681), afabd2bbf0b64c14a0517af4c7547efd (127.3.147.2:46169)
I20260812 09:40:11.447374 5264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7beb05c96af6460ebc53dfc0b3288341" candidate_uuid: "3d427e207f9449718f324cae6691ecac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afabd2bbf0b64c14a0517af4c7547efd"
I20260812 09:40:11.447489 5264 raft_consensus.cc:3037] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.447450 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7beb05c96af6460ebc53dfc0b3288341" candidate_uuid: "3d427e207f9449718f324cae6691ecac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b18f033a53642baa8049d6ceca7998f"
I20260812 09:40:11.447611 5106 raft_consensus.cc:3037] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:11.448547 5264 raft_consensus.cc:2454] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d427e207f9449718f324cae6691ecac in term 1.
I20260812 09:40:11.448824 5351 leader_election.cc:302] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d427e207f9449718f324cae6691ecac, afabd2bbf0b64c14a0517af4c7547efd; no voters:
I20260812 09:40:11.449013 5523 raft_consensus.cc:2781] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:11.449045 5106 raft_consensus.cc:2454] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d427e207f9449718f324cae6691ecac in term 1.
I20260812 09:40:11.449095 5523 raft_consensus.cc:686] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [term 1 LEADER]: Becoming Leader. State: Replica: 3d427e207f9449718f324cae6691ecac, State: Running, Role: LEADER
I20260812 09:40:11.449242 5523 consensus_queue.cc:237] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } }
I20260812 09:40:11.450078 5546 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: TxnStatusTablet state changed. Reason: New leader 3d427e207f9449718f324cae6691ecac. Latest consensus state: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.450243 5546 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:11.450800 5561 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 09:40:11.451004 5561 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 09:40:11.450875 4924 catalog_manager.cc:5495] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d427e207f9449718f324cae6691ecac (127.3.147.3). New cstate: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } health_report { overall_health: HEALTHY } } }
I20260812 09:40:11.463541 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.459218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:39933 (local address 127.0.0.1:58854)
0812 09:40:11.459376 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.459409 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.459508 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.459895 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.459897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.459905 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 09:40:11.460020 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.460024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.461085 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.461088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.461714 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.461722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.462375 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.462377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.462443 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.462485 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 09:40:11.463302 (+ 817us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 09:40:11.463316 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.463374 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.463440 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 09:40:11.463657 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.459374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.3.147.3:39933)
0812 09:40:11.459508 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.459513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.459528 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:40:11.459667 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.459670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.459738 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.459886 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:11.460131 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.460946 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.461848 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.462225 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.462365 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.462643 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:11.463193 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:11.463309 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.463408 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.463497 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 09:40:11.466895 5106 raft_consensus.cc:1261] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Refusing update from remote peer 3d427e207f9449718f324cae6691ecac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:11.467162 5264 raft_consensus.cc:1261] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Refusing update from remote peer 3d427e207f9449718f324cae6691ecac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:11.467509 5546 consensus_queue.cc:1035] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:11.467932 5546 consensus_queue.cc:1035] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac [LEADER]: Connected to new peer: Peer: permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:11.469167 5521 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: TxnStatusTablet state changed. Reason: New leader 3d427e207f9449718f324cae6691ecac. Latest consensus state: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.469270 5521 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:11.471163 5535 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: TxnStatusTablet state changed. Reason: New leader 3d427e207f9449718f324cae6691ecac. Latest consensus state: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.471279 5535 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:11.472313 5546 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.472421 5546 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:11.472579 5529 mvcc.cc:205] Tried to move back new op lower bound from 7317617096563519488 to 7317617096497442816. Current Snapshot: MvccSnapshot[applied={T|T < 7317617096563519488}]
I20260812 09:40:11.475509 5523 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.475616 5523 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P 3d427e207f9449718f324cae6691ecac: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:11.475767 5535 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.475878 5535 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P afabd2bbf0b64c14a0517af4c7547efd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:11.478036 5521 tablet_replica.cc:439] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d427e207f9449718f324cae6691ecac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 } } peers { permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 } } peers { permanent_uuid: "3d427e207f9449718f324cae6691ecac" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 39933 } } }
I20260812 09:40:11.478165 5521 tablet_replica.cc:442] T 7beb05c96af6460ebc53dfc0b3288341 P 0b18f033a53642baa8049d6ceca7998f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:11.482064 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.478565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58858 (local address 127.3.147.3:39933)
0812 09:40:11.478673 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.478677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.478691 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:11.478734 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.478737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.478859 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.479248 (+ 389us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:11.479344 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.480164 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.481000 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.481241 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.481352 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.481616 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:11.481692 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:11.481771 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.481891 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.481938 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 09:40:11.495072 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.490628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:39933 (local address 127.3.147.3:43029)
0812 09:40:11.490775 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:11.490792 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:11.490886 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:11.491142 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:11.491144 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:11.491150 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 09:40:11.491404 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.491411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.492611 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.492614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.493413 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:11.493422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.494111 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.494114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:11.494245 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.494288 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 09:40:11.494725 (+ 437us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 09:40:11.494733 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 09:40:11.494810 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 09:40:11.494882 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":40}
I20260812 09:40:11.495095 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.490615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:43029 (local address 127.3.147.3:39933)
0812 09:40:11.490754 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.490759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.490868 (+ 109us) server_negotiation.cc:407] Connection header received
0812 09:40:11.490983 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.490986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.491041 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.491136 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:11.491522 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.492493 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.493558 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.493962 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.494095 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.494435 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:11.494522 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:11.494629 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.494856 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.494913 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":68}
I20260812 09:40:11.503113 5264 raft_consensus.cc:1261] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Refusing update from remote peer 3d427e207f9449718f324cae6691ecac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:11.503219 5106 raft_consensus.cc:1261] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Refusing update from remote peer 3d427e207f9449718f324cae6691ecac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:11.503504 5546 consensus_queue.cc:1035] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [LEADER]: Connected to new peer: Peer: permanent_uuid: "afabd2bbf0b64c14a0517af4c7547efd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:11.503645 5523 consensus_queue.cc:1035] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b18f033a53642baa8049d6ceca7998f" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:11.509790 5531 mvcc.cc:205] Tried to move back new op lower bound from 7317617096713265152 to 7317617096107450368. Current Snapshot: MvccSnapshot[applied={T|T < 7317617096713265152}]
I20260812 09:40:11.511576 5528 mvcc.cc:205] Tried to move back new op lower bound from 7317617096713265152 to 7317617096107450368. Current Snapshot: MvccSnapshot[applied={T|T < 7317617096713265152}]
I20260812 09:40:11.555136 4921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:11.555359 4921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:11.557982 4921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47381f0454f0446b92b404c26876e9a2: Sending DeleteTablet for 3 replicas of tablet 489d11c548f44ddfa55dd802fcfedb7b
I20260812 09:40:11.558748 5081 tablet_service.cc:1514] Processing DeleteTablet for tablet 489d11c548f44ddfa55dd802fcfedb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:11 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35230
I20260812 09:40:11.558943 5538 tablet_replica.cc:330] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: stopping tablet replica
I20260812 09:40:11.559057 5538 raft_consensus.cc:2229] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:11.559181 5538 raft_consensus.cc:2258] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:11.559888 5538 ts_tablet_manager.cc:1905] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:11.562286 5243 tablet_service.cc:1514] Processing DeleteTablet for tablet 489d11c548f44ddfa55dd802fcfedb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:11 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:56164
I20260812 09:40:11.562506 5540 tablet_replica.cc:330] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: stopping tablet replica
I20260812 09:40:11.562641 5540 raft_consensus.cc:2229] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:11.562757 5540 raft_consensus.cc:2258] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:11.562747 5407 tablet_service.cc:1514] Processing DeleteTablet for tablet 489d11c548f44ddfa55dd802fcfedb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:11 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:58846
I20260812 09:40:11.562944 5539 tablet_replica.cc:330] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: stopping tablet replica
I20260812 09:40:11.563048 5539 raft_consensus.cc:2229] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:11.563372 5539 raft_consensus.cc:2258] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:11.563535 5538 ts_tablet_manager.cc:1918] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:11.563607 5538 log.cc:1196] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/489d11c548f44ddfa55dd802fcfedb7b
I20260812 09:40:11.564240 5538 ts_tablet_manager.cc:1939] T 489d11c548f44ddfa55dd802fcfedb7b P 0b18f033a53642baa8049d6ceca7998f: Deleting consensus metadata
I20260812 09:40:11.564412 5539 ts_tablet_manager.cc:1905] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:11.565160 4906 catalog_manager.cc:4841] TS 0b18f033a53642baa8049d6ceca7998f (127.3.147.1:36681): tablet 489d11c548f44ddfa55dd802fcfedb7b (table test-table [id=00b52235f1954672bb85cf16a51affb1]) successfully deleted
I20260812 09:40:11.566184 5540 ts_tablet_manager.cc:1905] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:11.567838 5539 ts_tablet_manager.cc:1918] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:11.567902 5539 log.cc:1196] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/489d11c548f44ddfa55dd802fcfedb7b
I20260812 09:40:11.568169 5539 ts_tablet_manager.cc:1939] T 489d11c548f44ddfa55dd802fcfedb7b P 3d427e207f9449718f324cae6691ecac: Deleting consensus metadata
I20260812 09:40:11.568885 4907 catalog_manager.cc:4841] TS 3d427e207f9449718f324cae6691ecac (127.3.147.3:39933): tablet 489d11c548f44ddfa55dd802fcfedb7b (table test-table [id=00b52235f1954672bb85cf16a51affb1]) successfully deleted
I20260812 09:40:11.569056 5540 ts_tablet_manager.cc:1918] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:11.569124 5540 log.cc:1196] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/489d11c548f44ddfa55dd802fcfedb7b
I20260812 09:40:11.569367 5540 ts_tablet_manager.cc:1939] T 489d11c548f44ddfa55dd802fcfedb7b P afabd2bbf0b64c14a0517af4c7547efd: Deleting consensus metadata
I20260812 09:40:11.570078 4906 catalog_manager.cc:4841] TS afabd2bbf0b64c14a0517af4c7547efd (127.3.147.2:46169): tablet 489d11c548f44ddfa55dd802fcfedb7b (table test-table [id=00b52235f1954672bb85cf16a51affb1]) successfully deleted
Aug 12 09:40:11 dist-test-slave-cbbz krb5kdc[4874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527611, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:11.588976 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.572606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35246 (local address 127.3.147.1:36681)
0812 09:40:11.572745 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.572750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.576541 (+ 3791us) server_negotiation.cc:407] Connection header received
0812 09:40:11.576666 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.576669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.576727 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.576818 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:11.577720 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.578535 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.579972 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.580349 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.580420 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.584889 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:11.584898 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:11.584914 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:11.584943 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:11.586838 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.587414 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.587419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.587421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.587500 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.587843 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.587845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.587846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.588083 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:11.588360 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.588685 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.588817 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 09:40:11.589955 5152 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35246
I20260812 09:40:11.599212 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:11.591821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58224 (local address 127.3.147.62:34813)
0812 09:40:11.591958 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:11.591962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:11.592045 (+ 83us) server_negotiation.cc:407] Connection header received
0812 09:40:11.592212 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:11.592216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:11.592272 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:11.592365 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:11.592994 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.593518 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.594386 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:11.594617 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:11.594726 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:11.596112 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:11.596116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:11.596118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:11.596167 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:11.597511 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.597961 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.597965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.597966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.598054 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:11.598354 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:11.598357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:11.598359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:11.598543 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:11.598685 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:11.598927 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 09:40:11.599050 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 09:40:11.599633 4921 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58224
I20260812 09:40:11.600970 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 5008
I20260812 09:40:11.612931 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 5169
I20260812 09:40:11.626399 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 5332
I20260812 09:40:11.637219 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 4887
2026-08-12T09:40:11Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2773 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:11 dist-test-slave-cbbz krb5kdc[5580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:11 dist-test-slave-cbbz krb5kdc[5580](info): set up 2 sockets
Aug 12 09:40:11 dist-test-slave-cbbz krb5kdc[5580](info): commencing operation
krb5kdc: starting...
W20260812 09:40:12.758333 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:12 dist-test-slave-cbbz krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:12.883282 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41007
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:41007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:12.992398 5595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:12.992668 5595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:12.992710 5595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:12.997007 5595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:12.997074 5595 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 09:40:12.997093 5595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:12.997112 5595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:12.997134 5595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:12.997149 5595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:13.002535 5595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:41007
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:13.003521 5595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:13.004566 5595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:13.012738 5603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:13.012903 5595 server_base.cc:1034] running on GCE node
W20260812 09:40:13.012746 5601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:13.012746 5600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:13.013602 5595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:13.014638 5595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:13.015831 5595 hybrid_clock.cc:648] HybridClock initialized: now 1786527613015805 us; error 36 us; skew 500 ppm
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:13.018592 5595 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:13.019433 5595 webserver.cc:460] Webserver started at http://127.3.147.62:41321/ using document root <none> and password file <none>
I20260812 09:40:13.019971 5595 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:13.020061 5595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:13.020319 5595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:13.022298 5595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "684e3aea33c043deacc318b7f25d0b70"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "29fce99cdd2c467c18238c6ed531416d"
server_key_iv: "9f8327c33bcfe5bec19da8f1db692caa"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.022855 5595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "684e3aea33c043deacc318b7f25d0b70"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "29fce99cdd2c467c18238c6ed531416d"
server_key_iv: "9f8327c33bcfe5bec19da8f1db692caa"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.026366 5595 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 09:40:13.028739 5610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:13.029901 5595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:13.030045 5595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "684e3aea33c043deacc318b7f25d0b70"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "29fce99cdd2c467c18238c6ed531416d"
server_key_iv: "9f8327c33bcfe5bec19da8f1db692caa"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.030155 5595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:13.059320 5595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:13.062116 5595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:13.062327 5595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:13.070302 5595 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:41007
I20260812 09:40:13.070340 5681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:41007 every 8 connection(s)
I20260812 09:40:13.071213 5595 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:13.073875 5682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:13.079262 5682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: Bootstrap starting.
I20260812 09:40:13.079481 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 5595
I20260812 09:40:13.079599 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:13.079873 3660 external_mini_cluster.cc:1428] Setting key 03d6c3b6f7066c563209a644ff1b6b47
I20260812 09:40:13.081933 5682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:13.082664 5682 log.cc:824] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:13.084803 5682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: No bootstrap required, opened a new log
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:13.089293 5682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684e3aea33c043deacc318b7f25d0b70" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41007 } }
I20260812 09:40:13.089541 5682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:13.089607 5682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 684e3aea33c043deacc318b7f25d0b70, State: Initialized, Role: FOLLOWER
I20260812 09:40:13.090015 5682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684e3aea33c043deacc318b7f25d0b70" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41007 } }
I20260812 09:40:13.090163 5682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:13.090231 5682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:13.090376 5682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:13.091212 5682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684e3aea33c043deacc318b7f25d0b70" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41007 } }
I20260812 09:40:13.091527 5682 leader_election.cc:302] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 684e3aea33c043deacc318b7f25d0b70; no voters:
I20260812 09:40:13.091869 5682 leader_election.cc:288] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:13.091935 5688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:13.092129 5688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [term 1 LEADER]: Becoming Leader. State: Replica: 684e3aea33c043deacc318b7f25d0b70, State: Running, Role: LEADER
I20260812 09:40:13.092491 5688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684e3aea33c043deacc318b7f25d0b70" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41007 } }
I20260812 09:40:13.093155 5682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:13.093636 5690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "684e3aea33c043deacc318b7f25d0b70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684e3aea33c043deacc318b7f25d0b70" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41007 } } }
I20260812 09:40:13.093779 5690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:13.094163 5689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 684e3aea33c043deacc318b7f25d0b70. Latest consensus state: current_term: 1 leader_uuid: "684e3aea33c043deacc318b7f25d0b70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "684e3aea33c043deacc318b7f25d0b70" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41007 } } }
I20260812 09:40:13.094280 5689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:13.095147 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.081255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55060 (local address 127.3.147.62:41007)
0812 09:40:13.081693 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.081708 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.081743 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:40:13.082493 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.082512 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.082892 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.083244 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:13.084070 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.085125 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.086293 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.086740 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.086865 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.089644 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:13.089652 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:13.089670 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:13.089705 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:13.091691 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.092267 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.092274 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.092280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.092448 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.092780 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.092784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.092788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.093194 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:13.093388 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.093893 (+ 505us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.094201 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":134,"threads_started":1}
W20260812 09:40:13.098296 5705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:40:13.098376 5705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:40:13.098433 5695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:13.099292 5695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:13.104717 5695 catalog_manager.cc:1347] Generated new cluster ID: d228a05a955e41d2a079aefb04618acd
I20260812 09:40:13.104780 5695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:13.120817 5695 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:13.121935 5695 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:13.127784 5695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: Generated new TSK 0
I20260812 09:40:13.128495 5695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:13.162951 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41007
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:13.273880 5714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:13.274116 5714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:13.274171 5714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:13.278214 5714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:13.278262 5714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:13.278301 5714 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 09:40:13.278348 5714 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:13.278419 5714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:13.283492 5714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:13.284512 5714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:13.285506 5714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:13.295042 5725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:13.295037 5726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:13.295235 5714 server_base.cc:1034] running on GCE node
W20260812 09:40:13.295035 5728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:13.295773 5714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:13.296566 5714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:13.297777 5714 hybrid_clock.cc:648] HybridClock initialized: now 1786527613297756 us; error 30 us; skew 500 ppm
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:13.300606 5714 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:13.301424 5714 webserver.cc:460] Webserver started at http://127.3.147.1:36239/ using document root <none> and password file <none>
I20260812 09:40:13.301990 5714 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:13.302083 5714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:13.302317 5714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:13.304364 5714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "5652e431dc7847b190b039d3dab856e1"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "bd6ed664d209c33b07d35be42bae701d"
server_key_iv: "cfa86ed8295f373994cfb86a4ca1387c"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.304929 5714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "5652e431dc7847b190b039d3dab856e1"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "bd6ed664d209c33b07d35be42bae701d"
server_key_iv: "cfa86ed8295f373994cfb86a4ca1387c"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.308557 5714 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:40:13.310827 5735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:13.311931 5714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:13.312079 5714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "5652e431dc7847b190b039d3dab856e1"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "bd6ed664d209c33b07d35be42bae701d"
server_key_iv: "cfa86ed8295f373994cfb86a4ca1387c"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.312223 5714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:13.343521 5714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:13.346313 5714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:13.346508 5714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:13.347908 5714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:13.347980 5714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:13.348050 5714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:13.348102 5714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:13.359668 5714 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:44287
I20260812 09:40:13.359947 5864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:44287 every 8 connection(s)
I20260812 09:40:13.360728 5714 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:13.362521 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.348811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:42295 (local address 127.3.147.62:41007)
0812 09:40:13.350208 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.350215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.350233 (+ 18us) server_negotiation.cc:407] Connection header received
0812 09:40:13.350991 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.350995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.351053 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.351161 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:13.352625 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.353215 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.354433 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.354713 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.354833 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.358729 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:13.358732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:13.358735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:13.358761 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:13.360455 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.361032 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.361035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.361036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.361150 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.361549 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.361553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.361555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.361786 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:13.361883 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.362307 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.362405 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300}
I20260812 09:40:13.363281 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.349222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41007 (local address 127.3.147.1:42295)
0812 09:40:13.349688 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.349739 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.350768 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.351347 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.351356 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.351861 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:13.352471 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.352488 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.353356 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.353360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.354289 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.354308 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.354894 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.354898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.355123 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.355797 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:13.355821 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:13.358567 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:13.360574 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.360579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.360591 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.360929 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.361269 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.361273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.361275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.361451 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.361896 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:13.361901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:13.362105 (+ 204us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.362256 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.362492 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":142,"threads_started":1}
I20260812 09:40:13.364619 5865 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41007
I20260812 09:40:13.364758 5865 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:13.365289 5865 heartbeater.cc:511] Master 127.3.147.62:41007 requested a full tablet report, sending...
I20260812 09:40:13.366876 5631 ts_manager.cc:194] Registered new tserver with Master: 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287)
I20260812 09:40:13.367983 5631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:42295
I20260812 09:40:13.369352 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 5714
I20260812 09:40:13.369479 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:13.369793 3660 external_mini_cluster.cc:1428] Setting key 9744fc4ef823e9112df971ce01845a37
I20260812 09:40:13.376978 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.366896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:49659 (local address 127.3.147.62:41007)
0812 09:40:13.367035 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.367039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.368409 (+ 1370us) server_negotiation.cc:407] Connection header received
0812 09:40:13.368510 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.368512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.368547 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.368650 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:13.370190 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.370875 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.372052 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.372314 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.372394 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.373598 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:13.373603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:13.373606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:13.373635 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:13.375093 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.375605 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.375607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.375609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.375677 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.376034 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.376037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.376038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.376310 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:13.376433 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.376714 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.376820 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":47}
I20260812 09:40:13.376986 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.366817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41007 (local address 127.3.147.1:49659)
0812 09:40:13.368323 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.368338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.368435 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.368706 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.368709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.368922 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:13.369046 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.369052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.371011 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.371026 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.371920 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.371936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.372433 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.372440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.372567 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.373008 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:13.373027 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:13.373460 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:13.375217 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.375223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.375228 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.375501 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.375765 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.375771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.375775 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.375883 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.376438 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:13.376442 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:13.376530 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.376650 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.376792 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1426,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:13.399153 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41007
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:13.512181 5877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:13.512439 5877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:13.512487 5877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:13.516571 5877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:13.516623 5877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:13.516661 5877 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 09:40:13.516675 5877 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:13.516705 5877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:13.521765 5877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:13.522679 5877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:13.523599 5877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:13.533026 5883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:13.533023 5885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:13.533142 5887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:13.533493 5877 server_base.cc:1034] running on GCE node
I20260812 09:40:13.533977 5877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:13.534705 5877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:13.535866 5877 hybrid_clock.cc:648] HybridClock initialized: now 1786527613535845 us; error 32 us; skew 500 ppm
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:13.538690 5877 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:13.539599 5877 webserver.cc:460] Webserver started at http://127.3.147.2:45589/ using document root <none> and password file <none>
I20260812 09:40:13.540110 5877 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:13.540241 5877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:13.540515 5877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:13.542511 5877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "d098028928d542cbafa7d0084423d64c"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "62affff07dc3ce64cdfe28647f17d64c"
server_key_iv: "19c2e5888a6b010a914c9eb5bc306172"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.543081 5877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "d098028928d542cbafa7d0084423d64c"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "62affff07dc3ce64cdfe28647f17d64c"
server_key_iv: "19c2e5888a6b010a914c9eb5bc306172"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.546875 5877 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 09:40:13.549566 5894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:13.550880 5877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 09:40:13.551122 5877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "d098028928d542cbafa7d0084423d64c"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "62affff07dc3ce64cdfe28647f17d64c"
server_key_iv: "19c2e5888a6b010a914c9eb5bc306172"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.551236 5877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:13.579849 5877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:13.582931 5877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:13.583151 5877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:13.584863 5877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:13.584937 5877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:13.585009 5877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:13.585048 5877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:13.597235 5877 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:35119
I20260812 09:40:13.597270 6023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:35119 every 8 connection(s)
I20260812 09:40:13.598394 5877 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:13.600311 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.586134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41007 (local address 127.3.147.2:46293)
0812 09:40:13.586630 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.586676 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.587516 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.588097 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.588106 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.588513 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:13.588961 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.588975 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.589844 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.589848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.590924 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.590935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.591580 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.591584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.591749 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.592338 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:13.592364 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:13.595103 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:13.597403 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.597408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.597421 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.597746 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.598051 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.598055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.598057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.598219 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.598637 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:13.598642 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:13.598975 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.599183 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.599503 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":141,"threads_started":1}
I20260812 09:40:13.600584 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.585582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:46293 (local address 127.3.147.62:41007)
0812 09:40:13.585731 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.585735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.586783 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 09:40:13.587742 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.587745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.587807 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.587913 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:13.589111 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.589718 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.591064 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.591436 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.591539 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.595290 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:13.595294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:13.595296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:13.595324 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:13.597261 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.597863 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.597866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.597867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.597948 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.598322 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.598324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.598325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.598541 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:13.598627 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.600307 (+ 1680us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.600441 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:40:13.601586 6024 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41007
I20260812 09:40:13.601756 6024 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:13.602319 6024 heartbeater.cc:511] Master 127.3.147.62:41007 requested a full tablet report, sending...
I20260812 09:40:13.603363 5632 ts_manager.cc:194] Registered new tserver with Master: d098028928d542cbafa7d0084423d64c (127.3.147.2:35119)
I20260812 09:40:13.604194 5632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:46293
I20260812 09:40:13.605718 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 5877
I20260812 09:40:13.605834 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:13.606110 3660 external_mini_cluster.cc:1428] Setting key 4885d5da57e9e44ee7d4024e553dfc66
I20260812 09:40:13.611497 6032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.603540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41007 (local address 127.3.147.2:50353)
0812 09:40:13.603840 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.603855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.603940 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.604734 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.604737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.604942 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:13.605054 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.605060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.605938 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.605940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.606544 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.606558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.607042 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.607045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.607153 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.607640 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:13.607651 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:13.607946 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:13.610049 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.610051 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.610053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.610260 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.610605 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.610607 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.610608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.610683 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.611100 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:13.611102 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:13.611188 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.611271 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.611371 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260812 09:40:13.611699 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.603589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:50353 (local address 127.3.147.62:41007)
0812 09:40:13.604536 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.604540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.604554 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:13.604587 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.604590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.604630 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.604714 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:13.605172 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.605788 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.606662 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.606919 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.607034 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.608086 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:13.608090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:13.608092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:13.608116 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:13.609926 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.610371 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.610377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.610381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.610498 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.610778 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.610781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.610783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.611011 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:13.611111 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.611358 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.611497 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877}
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:13.637413 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41007
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:13.753337 6036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:13.753561 6036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:13.753604 6036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:13.757707 6036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:13.757777 6036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:13.757843 6036 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 09:40:13.757875 6036 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:13.757932 6036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:13.763109 6036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:13.764098 6036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:13.765213 6036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:13.775574 6045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:13.775589 6042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:13.775840 6043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:13.776369 6036 server_base.cc:1034] running on GCE node
I20260812 09:40:13.776760 6036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:13.777348 6036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:13.778642 6036 hybrid_clock.cc:648] HybridClock initialized: now 1786527613778628 us; error 55 us; skew 500 ppm
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:13.785959 6036 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:13.786836 6036 webserver.cc:460] Webserver started at http://127.3.147.3:36767/ using document root <none> and password file <none>
I20260812 09:40:13.787405 6036 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:13.787497 6036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:13.787746 6036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:13.789850 6036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "9622a9a4efc04e49b5eccc2988d75ac7"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "dec9baf92a137f4282c85d532c13b20c"
server_key_iv: "d2751c8394acb09e177b3e35ae2f5cbe"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.790417 6036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "9622a9a4efc04e49b5eccc2988d75ac7"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "dec9baf92a137f4282c85d532c13b20c"
server_key_iv: "d2751c8394acb09e177b3e35ae2f5cbe"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.794363 6036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 09:40:13.797148 6053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:13.798476 6036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:40:13.798571 6036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "9622a9a4efc04e49b5eccc2988d75ac7"
format_stamp: "Formatted at 2026-08-12 09:40:13 on dist-test-slave-cbbz"
server_key: "dec9baf92a137f4282c85d532c13b20c"
server_key_iv: "d2751c8394acb09e177b3e35ae2f5cbe"
server_key_version: "encryptionkey@0"
I20260812 09:40:13.798662 6036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:13.811173 6036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:13.814458 6036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:13.814600 6036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:13.816237 6036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:13.816311 6036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:13.816376 6036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:13.816406 6036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:13.827721 6036 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42679
I20260812 09:40:13.827858 6184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42679 every 8 connection(s)
I20260812 09:40:13.828925 6036 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:13.830713 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.817093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:39421 (local address 127.3.147.62:41007)
0812 09:40:13.817267 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.817272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.818150 (+ 878us) server_negotiation.cc:407] Connection header received
0812 09:40:13.819513 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.819519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.819588 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.819654 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:13.821507 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.822127 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.823281 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.823527 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.823654 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.826655 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:13.826659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:13.826660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:13.826686 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:13.828408 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.829024 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.829028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.829030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.829125 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.829557 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.829562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.829565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.829807 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:13.829927 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.830417 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.830537 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 09:40:13.831426 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.817464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41007 (local address 127.3.147.3:39421)
0812 09:40:13.817974 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.818030 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.819241 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.820101 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.820111 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.820617 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:13.821319 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.821343 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.822271 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.822275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.823141 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.823151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.823710 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.823715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.823904 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.824533 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:13.824556 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:13.826473 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:13.828545 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.828554 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.828571 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.828893 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.829229 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.829235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.829241 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.829435 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.829924 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:13.829930 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:13.830171 (+ 241us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.830357 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.830666 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260812 09:40:13.832576 6185 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41007
I20260812 09:40:13.832719 6185 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:13.833351 6185 heartbeater.cc:511] Master 127.3.147.62:41007 requested a full tablet report, sending...
I20260812 09:40:13.834167 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 6036
I20260812 09:40:13.834296 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:13.834590 5631 ts_manager.cc:194] Registered new tserver with Master: 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679)
I20260812 09:40:13.834642 3660 external_mini_cluster.cc:1428] Setting key f4e390d300395568a8e2777906399826
I20260812 09:40:13.835343 5631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:39421
I20260812 09:40:13.836685 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:40:13.842090 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.834956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41007 (local address 127.3.147.3:54079)
0812 09:40:13.835356 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.835368 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.835486 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.835740 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.835743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.835938 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:13.836075 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.836082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.836983 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.836986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.837545 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.837553 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.838012 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.838016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.838113 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.838604 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:13.838624 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:13.838906 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:13.840609 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.840613 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.840615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.840820 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.841103 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:13.841106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:13.841108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:13.841199 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:13.841618 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:13.841621 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:13.841714 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.841844 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.841964 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":119,"threads_started":1}
I20260812 09:40:13.842254 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.834991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:54079 (local address 127.3.147.62:41007)
0812 09:40:13.835229 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.835232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.835448 (+ 216us) server_negotiation.cc:407] Connection header received
0812 09:40:13.835590 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.835593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.835639 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.835723 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:13.836217 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.836852 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.837668 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.837898 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.838006 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.839042 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:13.839045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:13.839047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:13.839071 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:13.840503 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.840930 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.840935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.840936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.841013 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.841306 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.841310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.841312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.841502 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:13.841626 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.841945 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.842079 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 09:40:13 dist-test-slave-cbbz krb5kdc[5580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:13.863041 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.855610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.3.147.62:41007)
0812 09:40:13.855742 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.855746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.855759 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:13.855792 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.855794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.855837 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.855902 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:13.856606 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.857079 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.857861 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.858058 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.858129 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.860502 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:13.860505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:13.860507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:13.860528 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:13.861716 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.862090 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.862092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.862094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.862164 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:13.862395 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:13.862397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:13.862398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:13.862546 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:13.862653 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.862843 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.862929 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 09:40:13.866027 5631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:13.868985 5631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:13.884052 6207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.877356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:44287 (local address 127.0.0.1:35868)
0812 09:40:13.878236 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.878292 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.878420 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.879321 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.879324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.879355 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:13.879627 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.879637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.881131 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.881134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.882453 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.882463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.883496 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.883498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.883582 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.883612 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.883764 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.883882 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":250,"spinlock_wait_cycles":5248,"thread_start_us":91,"threads_started":1}
I20260812 09:40:13.884061 6205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.877485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.3.147.1:44287)
0812 09:40:13.878921 (+ 1436us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.878929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.878961 (+ 32us) server_negotiation.cc:407] Connection header received
0812 09:40:13.879026 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.879030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.879169 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.879569 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:13.879731 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.880948 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.882587 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.883356 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.883492 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.883669 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.883810 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.883920 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1327,"thread_start_us":107,"threads_started":1}
I20260812 09:40:13.884205 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.878198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.3.147.3:42679)
0812 09:40:13.878927 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.878939 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.878960 (+ 21us) server_negotiation.cc:407] Connection header received
0812 09:40:13.879028 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.879032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.879172 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.879313 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:13.880056 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.880899 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.882851 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.883627 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.883746 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.883907 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.883953 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.884030 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":139,"threads_started":1}
I20260812 09:40:13.884073 6204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.877760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42679 (local address 127.0.0.1:52604)
0812 09:40:13.878633 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.878645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.878708 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.879406 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.879408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.879427 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:13.879933 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.879942 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.881142 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.881145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.882735 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.882742 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.883753 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.883759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.883839 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.883860 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.883918 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.883974 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":332,"spinlock_wait_cycles":8576,"thread_start_us":96,"threads_started":1}
I20260812 09:40:13.886281 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.877511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:35119 (local address 127.0.0.1:40290)
0812 09:40:13.878546 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:13.878561 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:13.878672 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:13.879314 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:13.879320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:13.879356 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:13.879625 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.879634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.881092 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.881098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.882716 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:13.882727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.885858 (+ 3131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.885862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:13.885972 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.885997 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 09:40:13.886087 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 09:40:13.886163 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":219,"spinlock_wait_cycles":6144,"thread_start_us":86,"threads_started":1}
I20260812 09:40:13.886400 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:13.878159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.3.147.2:35119)
0812 09:40:13.878511 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:13.878524 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:13.878652 (+ 128us) server_negotiation.cc:407] Connection header received
0812 09:40:13.878916 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:13.878923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:13.879135 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:13.879307 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:13.879774 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.880968 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.884385 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:13.885716 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:13.885855 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:13.886023 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:13.886159 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 09:40:13.886261 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":125,"threads_started":1}
I20260812 09:40:13.887308 6109 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce34399f8b94342867f261925dd1873 (DEFAULT_TABLE table=test-table [id=1047cde1ebdf4c848fbf5fc97db846f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:13.888497 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce34399f8b94342867f261925dd1873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:13.888916 5950 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce34399f8b94342867f261925dd1873 (DEFAULT_TABLE table=test-table [id=1047cde1ebdf4c848fbf5fc97db846f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:13.889864 5950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce34399f8b94342867f261925dd1873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:13.889721 5791 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce34399f8b94342867f261925dd1873 (DEFAULT_TABLE table=test-table [id=1047cde1ebdf4c848fbf5fc97db846f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:13.890659 5791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce34399f8b94342867f261925dd1873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:13.897037 6210 tablet_bootstrap.cc:492] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Bootstrap starting.
I20260812 09:40:13.897789 6213 tablet_bootstrap.cc:492] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Bootstrap starting.
I20260812 09:40:13.898466 6212 tablet_bootstrap.cc:492] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Bootstrap starting.
I20260812 09:40:13.899669 6210 tablet_bootstrap.cc:654] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:13.900002 6213 tablet_bootstrap.cc:654] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:13.900559 6212 tablet_bootstrap.cc:654] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:13.900557 6210 log.cc:824] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:13.900928 6213 log.cc:824] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:13.901358 6212 log.cc:824] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:13.902732 6210 tablet_bootstrap.cc:492] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: No bootstrap required, opened a new log
I20260812 09:40:13.902884 6213 tablet_bootstrap.cc:492] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: No bootstrap required, opened a new log
I20260812 09:40:13.902966 6210 ts_tablet_manager.cc:1397] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:40:13.903085 6213 ts_tablet_manager.cc:1397] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260812 09:40:13.905778 6212 tablet_bootstrap.cc:492] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: No bootstrap required, opened a new log
I20260812 09:40:13.906111 6212 ts_tablet_manager.cc:1397] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 09:40:13.906556 6213 raft_consensus.cc:348] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:13.906809 6213 raft_consensus.cc:374] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:13.906885 6213 raft_consensus.cc:729] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9622a9a4efc04e49b5eccc2988d75ac7, State: Initialized, Role: FOLLOWER
I20260812 09:40:13.906805 6210 raft_consensus.cc:348] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:13.907056 6210 raft_consensus.cc:374] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:13.907119 6210 raft_consensus.cc:729] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d098028928d542cbafa7d0084423d64c, State: Initialized, Role: FOLLOWER
I20260812 09:40:13.907415 6213 consensus_queue.cc:260] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:13.907601 6210 consensus_queue.cc:260] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:13.908183 6185 heartbeater.cc:503] Master 127.3.147.62:41007 was elected leader, sending a full tablet report...
I20260812 09:40:13.908527 6213 ts_tablet_manager.cc:1428] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260812 09:40:13.908614 6210 ts_tablet_manager.cc:1428] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:40:13.908943 6024 heartbeater.cc:503] Master 127.3.147.62:41007 was elected leader, sending a full tablet report...
I20260812 09:40:13.911649 6212 raft_consensus.cc:348] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:13.911926 6212 raft_consensus.cc:374] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:13.912037 6212 raft_consensus.cc:729] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5652e431dc7847b190b039d3dab856e1, State: Initialized, Role: FOLLOWER
I20260812 09:40:13.912484 6212 consensus_queue.cc:260] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:13.913182 5865 heartbeater.cc:503] Master 127.3.147.62:41007 was elected leader, sending a full tablet report...
I20260812 09:40:13.913422 6212 ts_tablet_manager.cc:1428] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
W20260812 09:40:14.079955 6186 tablet.cc:2369] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:40:14.099810 6025 tablet.cc:2369] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:40:14.112362 5866 tablet.cc:2369] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:40:14.178584 6222 raft_consensus.cc:482] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:14.178836 6222 raft_consensus.cc:504] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.180132 6222 leader_election.cc:288] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d098028928d542cbafa7d0084423d64c (127.3.147.2:35119), 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679)
I20260812 09:40:14.184278 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.180528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:39931 (local address 127.3.147.2:35119)
0812 09:40:14.180685 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.180690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.180945 (+ 255us) server_negotiation.cc:407] Connection header received
0812 09:40:14.181092 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.181094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.181144 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.181238 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:14.181574 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.182087 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.183244 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.183641 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.183724 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.183814 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.184040 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.184118 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 09:40:14.184274 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.180590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:47899 (local address 127.3.147.3:42679)
0812 09:40:14.180728 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.180734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.180995 (+ 261us) server_negotiation.cc:407] Connection header received
0812 09:40:14.181066 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.181072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.181125 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.181195 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:14.181543 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.182088 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.183268 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.183669 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.183798 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.183865 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.184064 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.184155 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:40:14.184306 6223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.180423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:35119 (local address 127.3.147.1:39931)
0812 09:40:14.180836 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.180849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.180983 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.181279 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.181282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.181300 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:14.181432 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.181437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.182209 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.182214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.183106 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.183116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.183773 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.183777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.183886 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.183917 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.183989 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.184088 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":26,"thread_start_us":148,"threads_started":1}
I20260812 09:40:14.184320 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.180437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42679 (local address 127.3.147.1:47899)
0812 09:40:14.180839 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.180851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.180951 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.181238 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.181240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.181257 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:14.181419 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.181426 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.182222 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.182226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.183095 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.183105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.183814 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.183818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.183919 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.183936 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.184029 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.184103 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":104,"spinlock_wait_cycles":1920,"thread_start_us":124,"threads_started":1}
I20260812 09:40:14.185034 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce34399f8b94342867f261925dd1873" candidate_uuid: "5652e431dc7847b190b039d3dab856e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c" is_pre_election: true
I20260812 09:40:14.185171 6130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce34399f8b94342867f261925dd1873" candidate_uuid: "5652e431dc7847b190b039d3dab856e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" is_pre_election: true
I20260812 09:40:14.185395 5973 raft_consensus.cc:2454] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5652e431dc7847b190b039d3dab856e1 in term 0.
I20260812 09:40:14.185405 6130 raft_consensus.cc:2454] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5652e431dc7847b190b039d3dab856e1 in term 0.
I20260812 09:40:14.185956 5737 leader_election.cc:302] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5652e431dc7847b190b039d3dab856e1, 9622a9a4efc04e49b5eccc2988d75ac7, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:14.186239 6222 raft_consensus.cc:2781] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:14.186348 6222 raft_consensus.cc:482] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:14.186412 6222 raft_consensus.cc:3037] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.187558 6222 raft_consensus.cc:504] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.187927 6222 leader_election.cc:288] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [CANDIDATE]: Term 1 election: Requested vote from peers d098028928d542cbafa7d0084423d64c (127.3.147.2:35119), 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679)
I20260812 09:40:14.188314 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce34399f8b94342867f261925dd1873" candidate_uuid: "5652e431dc7847b190b039d3dab856e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c"
I20260812 09:40:14.188318 6130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce34399f8b94342867f261925dd1873" candidate_uuid: "5652e431dc7847b190b039d3dab856e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9622a9a4efc04e49b5eccc2988d75ac7"
I20260812 09:40:14.188448 6130 raft_consensus.cc:3037] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.188448 5973 raft_consensus.cc:3037] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.189920 5973 raft_consensus.cc:2454] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5652e431dc7847b190b039d3dab856e1 in term 1.
I20260812 09:40:14.190284 5739 leader_election.cc:302] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5652e431dc7847b190b039d3dab856e1, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:14.190508 6130 raft_consensus.cc:2454] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5652e431dc7847b190b039d3dab856e1 in term 1.
I20260812 09:40:14.190697 6222 raft_consensus.cc:2781] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:14.190941 6222 raft_consensus.cc:686] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 1 LEADER]: Becoming Leader. State: Replica: 5652e431dc7847b190b039d3dab856e1, State: Running, Role: LEADER
I20260812 09:40:14.191316 6222 consensus_queue.cc:237] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.194823 5632 catalog_manager.cc:5495] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5652e431dc7847b190b039d3dab856e1 (127.3.147.1). New cstate: current_term: 1 leader_uuid: "5652e431dc7847b190b039d3dab856e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } health_report { overall_health: UNKNOWN } } }
Aug 12 09:40:14 dist-test-slave-cbbz krb5kdc[5580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527613, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:14.266225 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.257985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41007 (local address 127.0.0.1:55078)
0812 09:40:14.258356 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.258370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.258485 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.258740 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.258742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.259150 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:14.259260 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.259266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.259985 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.259987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.260600 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.260608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.261015 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.261018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.261087 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.261614 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:14.261626 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:14.263113 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:14.264744 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:14.264752 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:14.264757 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:14.264990 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:14.265252 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:14.265254 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:14.265256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:14.265335 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:14.265690 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:14.265716 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:14.265829 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.265923 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.266035 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":168,"threads_started":1}
I20260812 09:40:14.266223 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.258038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.3.147.62:41007)
0812 09:40:14.258202 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.258208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.258457 (+ 249us) server_negotiation.cc:407] Connection header received
0812 09:40:14.258570 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.258572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.258614 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.258695 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:14.259372 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.259887 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.260707 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.260915 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.260992 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.263243 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:14.263246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:14.263248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:14.263269 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:14.264627 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:14.265091 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:14.265094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:14.265095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:14.265170 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:14.265413 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:14.265415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:14.265416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:14.265576 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:14.265682 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.265985 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.266089 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":58}
I20260812 09:40:14.270145 5632 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:55078:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 09:40:14.270889 5632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:14.276070 5791 tablet_service.cc:1467] Processing CreateTablet for tablet 2ad64668223c4534bccead3750c4168e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb26718c4034ddc9e0059e39788068e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:14.276361 6109 tablet_service.cc:1467] Processing CreateTablet for tablet 2ad64668223c4534bccead3750c4168e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb26718c4034ddc9e0059e39788068e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:14.276360 5950 tablet_service.cc:1467] Processing CreateTablet for tablet 2ad64668223c4534bccead3750c4168e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cb26718c4034ddc9e0059e39788068e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:14.276909 5791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ad64668223c4534bccead3750c4168e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:14.277052 5950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ad64668223c4534bccead3750c4168e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:14.277057 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ad64668223c4534bccead3750c4168e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:14.279738 6212 tablet_bootstrap.cc:492] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: Bootstrap starting.
I20260812 09:40:14.279947 6213 tablet_bootstrap.cc:492] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: Bootstrap starting.
I20260812 09:40:14.280833 6213 tablet_bootstrap.cc:654] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:14.281071 6212 tablet_bootstrap.cc:654] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:14.281620 6210 tablet_bootstrap.cc:492] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: Bootstrap starting.
I20260812 09:40:14.282428 6213 tablet_bootstrap.cc:492] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: No bootstrap required, opened a new log
I20260812 09:40:14.282527 6213 ts_tablet_manager.cc:1397] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:14.282608 6212 tablet_bootstrap.cc:492] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: No bootstrap required, opened a new log
I20260812 09:40:14.282618 6210 tablet_bootstrap.cc:654] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:14.282696 6212 ts_tablet_manager.cc:1397] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:14.283053 6213 raft_consensus.cc:348] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.283093 6212 raft_consensus.cc:348] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.283185 6213 raft_consensus.cc:374] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:14.283224 6213 raft_consensus.cc:729] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9622a9a4efc04e49b5eccc2988d75ac7, State: Initialized, Role: FOLLOWER
I20260812 09:40:14.283232 6212 raft_consensus.cc:374] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:14.283268 6212 raft_consensus.cc:729] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5652e431dc7847b190b039d3dab856e1, State: Initialized, Role: FOLLOWER
I20260812 09:40:14.283355 6213 consensus_queue.cc:260] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.283422 6212 consensus_queue.cc:260] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.283687 6213 ts_tablet_manager.cc:1428] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 09:40:14.283631 6220 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.283715 6212 ts_tablet_manager.cc:1428] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:14.283769 6220 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:14.284039 6222 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.284201 6222 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:14.284363 6210 tablet_bootstrap.cc:492] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: No bootstrap required, opened a new log
I20260812 09:40:14.284456 6210 ts_tablet_manager.cc:1397] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:14.284966 6210 raft_consensus.cc:348] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.285079 6210 raft_consensus.cc:374] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:14.285115 6210 raft_consensus.cc:729] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d098028928d542cbafa7d0084423d64c, State: Initialized, Role: FOLLOWER
I20260812 09:40:14.285254 6210 consensus_queue.cc:260] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.285547 6221 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.285718 6221 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:14.285559 6210 ts_tablet_manager.cc:1428] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:14.308777 6220 raft_consensus.cc:482] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:14.308900 6220 raft_consensus.cc:504] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.309942 6220 leader_election.cc:288] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287), d098028928d542cbafa7d0084423d64c (127.3.147.2:35119)
I20260812 09:40:14.314647 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.310167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:44287 (local address 127.3.147.3:39335)
0812 09:40:14.310292 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.310308 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.310398 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.310672 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.310678 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.310704 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:14.310864 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.310870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.311898 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.311900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.313256 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.313268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.314182 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.314186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.314295 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.314338 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.314430 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.314508 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":49,"spinlock_wait_cycles":14592}
I20260812 09:40:14.314651 6205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.310304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:39335 (local address 127.3.147.1:44287)
0812 09:40:14.310425 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.310431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.310446 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:40:14.310504 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.310506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.310548 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.310617 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:14.310988 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.311768 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.313398 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.314023 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.314123 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.314189 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.314447 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.314524 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:40:14.314818 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.310292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:42685 (local address 127.3.147.2:35119)
0812 09:40:14.310419 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.310424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.310748 (+ 324us) server_negotiation.cc:407] Connection header received
0812 09:40:14.310852 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.310854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.310894 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.310990 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:14.311256 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.311978 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.313414 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.314015 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.314124 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.314189 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.314636 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.314709 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 09:40:14.314862 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.310167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:35119 (local address 127.3.147.3:42685)
0812 09:40:14.310625 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.310639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.310751 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.311013 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.311016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.311032 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:14.311151 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.311155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.312072 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.312075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.313307 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.313318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.314332 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.314337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.314447 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.314473 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.314616 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.314693 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":128,"spinlock_wait_cycles":1408,"thread_start_us":105,"threads_started":1}
I20260812 09:40:14.315379 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ad64668223c4534bccead3750c4168e" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c" is_pre_election: true
I20260812 09:40:14.315377 5813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ad64668223c4534bccead3750c4168e" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5652e431dc7847b190b039d3dab856e1" is_pre_election: true
I20260812 09:40:14.315544 5973 raft_consensus.cc:2454] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 0.
I20260812 09:40:14.315678 5813 raft_consensus.cc:2454] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 0.
I20260812 09:40:14.315922 6056 leader_election.cc:302] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9622a9a4efc04e49b5eccc2988d75ac7, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:14.316272 6220 raft_consensus.cc:2781] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:14.316359 6220 raft_consensus.cc:482] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:14.316419 6220 raft_consensus.cc:3037] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.317579 6220 raft_consensus.cc:504] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.317890 6220 leader_election.cc:288] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 election: Requested vote from peers 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287), d098028928d542cbafa7d0084423d64c (127.3.147.2:35119)
I20260812 09:40:14.318281 5813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ad64668223c4534bccead3750c4168e" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5652e431dc7847b190b039d3dab856e1"
I20260812 09:40:14.318328 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ad64668223c4534bccead3750c4168e" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c"
I20260812 09:40:14.318409 5813 raft_consensus.cc:3037] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.318480 5973 raft_consensus.cc:3037] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.319412 5813 raft_consensus.cc:2454] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 1.
I20260812 09:40:14.319412 5973 raft_consensus.cc:2454] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 1.
I20260812 09:40:14.319720 6054 leader_election.cc:302] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5652e431dc7847b190b039d3dab856e1, 9622a9a4efc04e49b5eccc2988d75ac7; no voters:
I20260812 09:40:14.319897 6220 raft_consensus.cc:2781] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:14.320111 6220 raft_consensus.cc:686] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 LEADER]: Becoming Leader. State: Replica: 9622a9a4efc04e49b5eccc2988d75ac7, State: Running, Role: LEADER
I20260812 09:40:14.320528 6220 consensus_queue.cc:237] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.322715 6238 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: TxnStatusTablet state changed. Reason: New leader 9622a9a4efc04e49b5eccc2988d75ac7. Latest consensus state: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.322839 6238 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:14.323136 6240 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 09:40:14.323357 6240 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 09:40:14.324075 5632 catalog_manager.cc:5495] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } health_report { overall_health: HEALTHY } } }
I20260812 09:40:14.347589 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.343579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42679 (local address 127.0.0.1:52618)
0812 09:40:14.343713 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.343730 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.343831 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.344199 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.344202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.344218 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 09:40:14.344318 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.344323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.345329 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.345331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.345930 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.345938 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.346497 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.346499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.346569 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.346613 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.347341 (+ 728us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.347353 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.347405 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.347481 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 09:40:14.347608 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.343673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.3.147.3:42679)
0812 09:40:14.343794 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.343798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.343814 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:40:14.343967 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.343971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.344034 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.344165 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:14.344424 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.345208 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.346036 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.346380 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.346483 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.346748 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.347248 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.347358 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.347417 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.347471 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:40:14.352725 5813 raft_consensus.cc:1261] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Refusing update from remote peer 9622a9a4efc04e49b5eccc2988d75ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:14.352727 5973 raft_consensus.cc:1261] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Refusing update from remote peer 9622a9a4efc04e49b5eccc2988d75ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:14.353441 6220 consensus_queue.cc:1035] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:14.353727 6238 consensus_queue.cc:1035] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:14.357086 6222 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: TxnStatusTablet state changed. Reason: New leader 9622a9a4efc04e49b5eccc2988d75ac7. Latest consensus state: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.357218 6222 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:14.358243 6221 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: TxnStatusTablet state changed. Reason: New leader 9622a9a4efc04e49b5eccc2988d75ac7. Latest consensus state: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.358373 6221 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:14.360033 6220 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.360170 6220 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:14.360339 6243 mvcc.cc:205] Tried to move back new op lower bound from 7317617108381884416 to 7317617108259483648. Current Snapshot: MvccSnapshot[applied={T|T < 7317617108381884416}]
I20260812 09:40:14.361696 6222 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.361815 6222 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P 5652e431dc7847b190b039d3dab856e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:14.362028 6238 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.362123 6238 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P 9622a9a4efc04e49b5eccc2988d75ac7: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:14.363448 6221 tablet_replica.cc:439] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } } }
I20260812 09:40:14.363550 6221 tablet_replica.cc:442] T 2ad64668223c4534bccead3750c4168e P d098028928d542cbafa7d0084423d64c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:14.363734 6245 mvcc.cc:205] Tried to move back new op lower bound from 7317617108381884416 to 7317617108259483648. Current Snapshot: MvccSnapshot[applied={T|T < 7317617108381884416}]
I20260812 09:40:14.367808 6244 mvcc.cc:205] Tried to move back new op lower bound from 7317617108381884416 to 7317617108259483648. Current Snapshot: MvccSnapshot[applied={T|T < 7317617108381884416}]
I20260812 09:40:14.374750 6205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.369877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.3.147.1:44287)
0812 09:40:14.369982 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.369986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.370280 (+ 294us) server_negotiation.cc:407] Connection header received
0812 09:40:14.370501 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.370504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.370559 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.370622 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:14.370970 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.371766 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.373042 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.373374 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.373445 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.373819 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.374275 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.374362 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.374570 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.374628 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 09:40:14.386406 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.382615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42679 (local address 127.3.147.1:34983)
0812 09:40:14.382935 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.382949 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.383038 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.383290 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.383292 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.383300 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 09:40:14.383381 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.383385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.384390 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.384392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.385076 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.385086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.385682 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.385685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.385746 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.385788 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.386131 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.386138 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.386208 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.386272 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":125,"threads_started":1}
I20260812 09:40:14.386420 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.382740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:34983 (local address 127.3.147.3:42679)
0812 09:40:14.382862 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.382867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.383020 (+ 153us) server_negotiation.cc:407] Connection header received
0812 09:40:14.383128 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.383139 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.383191 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.383276 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:14.383477 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.384287 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.385193 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.385559 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.385657 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.385943 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.386028 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.386103 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.386239 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.386290 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:40:14.394013 5973 raft_consensus.cc:1261] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Refusing update from remote peer 5652e431dc7847b190b039d3dab856e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:14.394227 6130 raft_consensus.cc:1261] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Refusing update from remote peer 5652e431dc7847b190b039d3dab856e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:14.394526 6225 consensus_queue.cc:1035] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:14.394701 6222 consensus_queue.cc:1035] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:14.418907 6205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.414885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:44589 (local address 127.3.147.1:44287)
0812 09:40:14.415024 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.415029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.415259 (+ 230us) server_negotiation.cc:407] Connection header received
0812 09:40:14.415386 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.415389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.415447 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.415527 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:14.415839 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.416578 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.417523 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.417773 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.417891 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.418375 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.418459 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.418511 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.418715 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.418781 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:40:14.419529 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.414815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:44287 (local address 127.3.147.3:44589)
0812 09:40:14.415164 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.415180 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.415286 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.415542 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.415545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.415553 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 09:40:14.415723 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.415730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.416699 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.416703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.417397 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.417407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.418014 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.418017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.418125 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.418182 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.418579 (+ 397us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.418584 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.419200 (+ 616us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.419284 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":135,"threads_started":1}
I20260812 09:40:14.444842 5631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:14.445053 5631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:14.447860 5631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: Sending DeleteTablet for 3 replicas of tablet 2ce34399f8b94342867f261925dd1873
I20260812 09:40:14.448731 5950 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce34399f8b94342867f261925dd1873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:14 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:40290
I20260812 09:40:14.448855 6109 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce34399f8b94342867f261925dd1873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:14 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:52604
I20260812 09:40:14.449338 6268 tablet_replica.cc:330] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: stopping tablet replica
I20260812 09:40:14.449664 6268 raft_consensus.cc:2229] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:14.449841 6268 raft_consensus.cc:2258] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:14.450241 5631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:14.450649 5631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:14.451203 6268 ts_tablet_manager.cc:1905] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:14.453120 5791 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce34399f8b94342867f261925dd1873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:14 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35868
I20260812 09:40:14.454725 6268 ts_tablet_manager.cc:1918] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:14.454824 6268 log.cc:1196] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/2ce34399f8b94342867f261925dd1873
I20260812 09:40:14.455159 6268 ts_tablet_manager.cc:1939] T 2ce34399f8b94342867f261925dd1873 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting consensus metadata
I20260812 09:40:14.456446 5613 catalog_manager.cc:4841] TS 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679): tablet 2ce34399f8b94342867f261925dd1873 (table test-table [id=1047cde1ebdf4c848fbf5fc97db846f0]) successfully deleted
I20260812 09:40:14.457609 5791 tablet_service.cc:1467] Processing CreateTablet for tablet 291aa3d37c9149b9a9457b32cb5e55d7 (DEFAULT_TABLE table=test-table [id=0946e83639c340f9842192ea75dae84d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:14.457631 6109 tablet_service.cc:1467] Processing CreateTablet for tablet 291aa3d37c9149b9a9457b32cb5e55d7 (DEFAULT_TABLE table=test-table [id=0946e83639c340f9842192ea75dae84d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:14.457880 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291aa3d37c9149b9a9457b32cb5e55d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:14.457899 5791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291aa3d37c9149b9a9457b32cb5e55d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:14.460063 6213 tablet_bootstrap.cc:492] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: Bootstrap starting.
I20260812 09:40:14.460803 5950 tablet_service.cc:1467] Processing CreateTablet for tablet 291aa3d37c9149b9a9457b32cb5e55d7 (DEFAULT_TABLE table=test-table [id=0946e83639c340f9842192ea75dae84d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:14.461086 5950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291aa3d37c9149b9a9457b32cb5e55d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:14.461179 6213 tablet_bootstrap.cc:654] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:14.461429 6212 tablet_bootstrap.cc:492] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: Bootstrap starting.
I20260812 09:40:14.462503 6212 tablet_bootstrap.cc:654] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:14.463442 6210 tablet_bootstrap.cc:492] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: Bootstrap starting.
I20260812 09:40:14.464576 6210 tablet_bootstrap.cc:654] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:14.464902 6212 tablet_bootstrap.cc:492] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: No bootstrap required, opened a new log
I20260812 09:40:14.464983 6212 ts_tablet_manager.cc:1397] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 09:40:14.465469 6212 raft_consensus.cc:348] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.465600 6212 raft_consensus.cc:374] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:14.465657 6212 raft_consensus.cc:729] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5652e431dc7847b190b039d3dab856e1, State: Initialized, Role: FOLLOWER
I20260812 09:40:14.465792 6212 consensus_queue.cc:260] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.466059 6212 ts_tablet_manager.cc:1428] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:14.466471 6210 tablet_bootstrap.cc:492] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: No bootstrap required, opened a new log
I20260812 09:40:14.466554 6210 ts_tablet_manager.cc:1397] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:14.466964 6210 raft_consensus.cc:348] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.467062 6210 raft_consensus.cc:374] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:14.467092 6210 raft_consensus.cc:729] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d098028928d542cbafa7d0084423d64c, State: Initialized, Role: FOLLOWER
I20260812 09:40:14.467201 6210 consensus_queue.cc:260] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.467445 6210 ts_tablet_manager.cc:1428] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:14.469389 6267 tablet_replica.cc:330] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: stopping tablet replica
I20260812 09:40:14.469763 6267 raft_consensus.cc:2229] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:14.469997 6267 raft_consensus.cc:2258] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:14.470366 6269 tablet_replica.cc:330] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: stopping tablet replica
I20260812 09:40:14.470683 6269 raft_consensus.cc:2229] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:14.471050 6269 raft_consensus.cc:2258] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:14.472826 6267 ts_tablet_manager.cc:1905] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:14.473155 6269 ts_tablet_manager.cc:1905] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:14.476074 6267 ts_tablet_manager.cc:1918] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:14.476207 6267 log.cc:1196] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/2ce34399f8b94342867f261925dd1873
I20260812 09:40:14.476593 6267 ts_tablet_manager.cc:1939] T 2ce34399f8b94342867f261925dd1873 P d098028928d542cbafa7d0084423d64c: Deleting consensus metadata
I20260812 09:40:14.477057 6213 tablet_bootstrap.cc:492] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: No bootstrap required, opened a new log
I20260812 09:40:14.477151 6213 ts_tablet_manager.cc:1397] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260812 09:40:14.477669 6213 raft_consensus.cc:348] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.477802 6213 raft_consensus.cc:374] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:14.477833 6213 raft_consensus.cc:729] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9622a9a4efc04e49b5eccc2988d75ac7, State: Initialized, Role: FOLLOWER
I20260812 09:40:14.477911 5615 catalog_manager.cc:4841] TS d098028928d542cbafa7d0084423d64c (127.3.147.2:35119): tablet 2ce34399f8b94342867f261925dd1873 (table test-table [id=1047cde1ebdf4c848fbf5fc97db846f0]) successfully deleted
I20260812 09:40:14.477958 6213 consensus_queue.cc:260] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.478235 6213 ts_tablet_manager.cc:1428] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:14.480185 6269 ts_tablet_manager.cc:1918] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:14.480347 6269 log.cc:1196] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/2ce34399f8b94342867f261925dd1873
I20260812 09:40:14.480751 6269 ts_tablet_manager.cc:1939] T 2ce34399f8b94342867f261925dd1873 P 5652e431dc7847b190b039d3dab856e1: Deleting consensus metadata
I20260812 09:40:14.482039 5612 catalog_manager.cc:4841] TS 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287): tablet 2ce34399f8b94342867f261925dd1873 (table test-table [id=1047cde1ebdf4c848fbf5fc97db846f0]) successfully deleted
I20260812 09:40:14.564231 6265 raft_consensus.cc:482] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:14.564455 6265 raft_consensus.cc:504] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.564960 6265 leader_election.cc:288] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287), d098028928d542cbafa7d0084423d64c (127.3.147.2:35119)
I20260812 09:40:14.565423 5813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291aa3d37c9149b9a9457b32cb5e55d7" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5652e431dc7847b190b039d3dab856e1" is_pre_election: true
I20260812 09:40:14.565536 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291aa3d37c9149b9a9457b32cb5e55d7" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c" is_pre_election: true
I20260812 09:40:14.565608 5813 raft_consensus.cc:2454] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 0.
I20260812 09:40:14.565676 5973 raft_consensus.cc:2454] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 0.
I20260812 09:40:14.566010 6056 leader_election.cc:302] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9622a9a4efc04e49b5eccc2988d75ac7, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:14.566221 6265 raft_consensus.cc:2781] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:14.566274 6265 raft_consensus.cc:482] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:14.566313 6265 raft_consensus.cc:3037] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.567605 6265 raft_consensus.cc:504] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.568006 6265 leader_election.cc:288] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 election: Requested vote from peers 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287), d098028928d542cbafa7d0084423d64c (127.3.147.2:35119)
I20260812 09:40:14.568490 5813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291aa3d37c9149b9a9457b32cb5e55d7" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5652e431dc7847b190b039d3dab856e1"
I20260812 09:40:14.568511 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291aa3d37c9149b9a9457b32cb5e55d7" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c"
I20260812 09:40:14.568660 5973 raft_consensus.cc:3037] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.568657 5813 raft_consensus.cc:3037] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:14.569635 5973 raft_consensus.cc:2454] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 1.
I20260812 09:40:14.569977 6056 leader_election.cc:302] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9622a9a4efc04e49b5eccc2988d75ac7, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:14.570180 6265 raft_consensus.cc:2781] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:14.570319 6265 raft_consensus.cc:686] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 LEADER]: Becoming Leader. State: Replica: 9622a9a4efc04e49b5eccc2988d75ac7, State: Running, Role: LEADER
I20260812 09:40:14.570444 5813 raft_consensus.cc:2454] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 1.
I20260812 09:40:14.570473 6265 consensus_queue.cc:237] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:14.572530 5631 catalog_manager.cc:5495] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } health_report { overall_health: HEALTHY } } }
I20260812 09:40:14.607525 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.604334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.3.147.3:42679)
0812 09:40:14.604662 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.604665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.604693 (+ 28us) server_negotiation.cc:407] Connection header received
0812 09:40:14.604729 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.604731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.604767 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.604863 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:14.605094 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.605617 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.606476 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.606707 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.606819 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.607076 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.607155 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.607238 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.607334 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.607379 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240}
I20260812 09:40:14.616786 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.613404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42679 (local address 127.3.147.3:59467)
0812 09:40:14.613565 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:14.613579 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:14.613656 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:14.614088 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:14.614090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:14.614096 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 09:40:14.614183 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.614188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.614866 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.614872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.615595 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:14.615626 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.616036 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.616039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:14.616124 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.616189 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.616511 (+ 322us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.616519 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 09:40:14.616585 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 09:40:14.616662 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 09:40:14.616786 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:14.613812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:59467 (local address 127.3.147.3:42679)
0812 09:40:14.613895 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:14.613899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:14.613909 (+ 10us) server_negotiation.cc:407] Connection header received
0812 09:40:14.613950 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:14.613952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:14.613986 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:14.614070 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:14.614289 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.614761 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.615719 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:14.615932 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:14.616036 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:14.616349 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:14.616445 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:14.616509 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:14.616595 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 09:40:14.616639 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 09:40:14.622833 5812 raft_consensus.cc:1261] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Refusing update from remote peer 9622a9a4efc04e49b5eccc2988d75ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:14.623281 5973 raft_consensus.cc:1261] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Refusing update from remote peer 9622a9a4efc04e49b5eccc2988d75ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:14.623718 6265 consensus_queue.cc:1035] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:14.623917 6220 consensus_queue.cc:1035] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 09:40:14.643782 6260 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 09:40:16.204695 5632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:16.204875 5632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:16.207305 5632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: Sending DeleteTablet for 3 replicas of tablet 291aa3d37c9149b9a9457b32cb5e55d7
I20260812 09:40:16.207858 5950 tablet_service.cc:1514] Processing DeleteTablet for tablet 291aa3d37c9149b9a9457b32cb5e55d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:16 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:40290
I20260812 09:40:16.207875 5791 tablet_service.cc:1514] Processing DeleteTablet for tablet 291aa3d37c9149b9a9457b32cb5e55d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:16 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35868
I20260812 09:40:16.207927 6108 tablet_service.cc:1514] Processing DeleteTablet for tablet 291aa3d37c9149b9a9457b32cb5e55d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:16 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:52604
I20260812 09:40:16.208271 6305 tablet_replica.cc:330] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: stopping tablet replica
I20260812 09:40:16.208338 6304 tablet_replica.cc:330] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: stopping tablet replica
I20260812 09:40:16.208451 6305 raft_consensus.cc:2229] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:16.208492 6304 raft_consensus.cc:2229] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:16.208630 6304 raft_consensus.cc:2258] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:16.208841 6305 raft_consensus.cc:2258] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:16.208894 6302 tablet_replica.cc:330] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: stopping tablet replica
I20260812 09:40:16.209087 6302 raft_consensus.cc:2229] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:16.209244 6302 raft_consensus.cc:2258] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:16.209337 6304 ts_tablet_manager.cc:1905] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:16.210177 6305 ts_tablet_manager.cc:1905] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:16.212266 6304 ts_tablet_manager.cc:1918] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 09:40:16.212349 6304 log.cc:1196] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/291aa3d37c9149b9a9457b32cb5e55d7
I20260812 09:40:16.212474 6302 ts_tablet_manager.cc:1905] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:16.212649 6304 ts_tablet_manager.cc:1939] T 291aa3d37c9149b9a9457b32cb5e55d7 P d098028928d542cbafa7d0084423d64c: Deleting consensus metadata
I20260812 09:40:16.213150 6305 ts_tablet_manager.cc:1918] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 09:40:16.213235 6305 log.cc:1196] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/291aa3d37c9149b9a9457b32cb5e55d7
I20260812 09:40:16.213359 5615 catalog_manager.cc:4841] TS d098028928d542cbafa7d0084423d64c (127.3.147.2:35119): tablet 291aa3d37c9149b9a9457b32cb5e55d7 (table test-table [id=0946e83639c340f9842192ea75dae84d]) successfully deleted
I20260812 09:40:16.213603 6305 ts_tablet_manager.cc:1939] T 291aa3d37c9149b9a9457b32cb5e55d7 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting consensus metadata
I20260812 09:40:16.214299 5613 catalog_manager.cc:4841] TS 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679): tablet 291aa3d37c9149b9a9457b32cb5e55d7 (table test-table [id=0946e83639c340f9842192ea75dae84d]) successfully deleted
I20260812 09:40:16.214994 6302 ts_tablet_manager.cc:1918] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 09:40:16.215077 6302 log.cc:1196] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/291aa3d37c9149b9a9457b32cb5e55d7
I20260812 09:40:16.215289 6302 ts_tablet_manager.cc:1939] T 291aa3d37c9149b9a9457b32cb5e55d7 P 5652e431dc7847b190b039d3dab856e1: Deleting consensus metadata
I20260812 09:40:16.215860 5612 catalog_manager.cc:4841] TS 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287): tablet 291aa3d37c9149b9a9457b32cb5e55d7 (table test-table [id=0946e83639c340f9842192ea75dae84d]) successfully deleted
Aug 12 09:40:16 dist-test-slave-cbbz krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527616, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 09:40:16 dist-test-slave-cbbz krb5kdc[5580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527616, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:16.240782 6314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:16.231268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.3.147.62:41007)
0812 09:40:16.231584 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:16.231595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:16.231613 (+ 18us) server_negotiation.cc:407] Connection header received
0812 09:40:16.231798 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:16.231802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:16.231857 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:16.231954 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:16.232838 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:16.233407 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:16.234297 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:16.234535 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:16.234645 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:16.237274 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:16.237277 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:16.237279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:16.237303 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:16.238835 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:16.239291 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:16.239296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:16.239298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:16.239385 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:16.239740 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:16.239744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:16.239745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:16.240000 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:16.240169 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:16.240458 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 09:40:16.240587 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":105,"threads_started":1}
I20260812 09:40:16.243400 5632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:16.243862 5632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:16.248636 5791 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1f5eb8992f486db6ed515424f4eeb7 (DEFAULT_TABLE table=test-table [id=2eacc153aa3a4f7784be75758e1a6744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:16.248708 6108 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1f5eb8992f486db6ed515424f4eeb7 (DEFAULT_TABLE table=test-table [id=2eacc153aa3a4f7784be75758e1a6744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:16.248711 5950 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1f5eb8992f486db6ed515424f4eeb7 (DEFAULT_TABLE table=test-table [id=2eacc153aa3a4f7784be75758e1a6744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:16.248960 6108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1f5eb8992f486db6ed515424f4eeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:16.248960 5791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1f5eb8992f486db6ed515424f4eeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:16.249055 5950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1f5eb8992f486db6ed515424f4eeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:16.251724 6318 tablet_bootstrap.cc:492] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: Bootstrap starting.
I20260812 09:40:16.251844 6315 tablet_bootstrap.cc:492] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: Bootstrap starting.
I20260812 09:40:16.252319 6317 tablet_bootstrap.cc:492] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: Bootstrap starting.
I20260812 09:40:16.252723 6315 tablet_bootstrap.cc:654] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:16.252723 6318 tablet_bootstrap.cc:654] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:16.253715 6317 tablet_bootstrap.cc:654] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:16.253928 6318 tablet_bootstrap.cc:492] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: No bootstrap required, opened a new log
I20260812 09:40:16.253928 6315 tablet_bootstrap.cc:492] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: No bootstrap required, opened a new log
I20260812 09:40:16.254024 6318 ts_tablet_manager.cc:1397] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 09:40:16.254024 6315 ts_tablet_manager.cc:1397] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:16.254491 6318 raft_consensus.cc:348] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.254477 6315 raft_consensus.cc:348] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.254599 6315 raft_consensus.cc:374] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:16.254659 6315 raft_consensus.cc:729] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d098028928d542cbafa7d0084423d64c, State: Initialized, Role: FOLLOWER
I20260812 09:40:16.254695 6318 raft_consensus.cc:374] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:16.254786 6318 raft_consensus.cc:729] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5652e431dc7847b190b039d3dab856e1, State: Initialized, Role: FOLLOWER
I20260812 09:40:16.254887 6315 consensus_queue.cc:260] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.255007 6318 consensus_queue.cc:260] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.255352 6315 ts_tablet_manager.cc:1428] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 09:40:16.255431 6317 tablet_bootstrap.cc:492] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: No bootstrap required, opened a new log
I20260812 09:40:16.255522 6317 ts_tablet_manager.cc:1397] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:16.255528 6318 ts_tablet_manager.cc:1428] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 09:40:16.255941 6317 raft_consensus.cc:348] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.256050 6317 raft_consensus.cc:374] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:16.256074 6317 raft_consensus.cc:729] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9622a9a4efc04e49b5eccc2988d75ac7, State: Initialized, Role: FOLLOWER
I20260812 09:40:16.256242 6317 consensus_queue.cc:260] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.256538 6317 ts_tablet_manager.cc:1428] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:16.448096 6279 raft_consensus.cc:482] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:16.448300 6279 raft_consensus.cc:504] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.448765 6279 leader_election.cc:288] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287), d098028928d542cbafa7d0084423d64c (127.3.147.2:35119)
I20260812 09:40:16.449275 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1f5eb8992f486db6ed515424f4eeb7" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c" is_pre_election: true
I20260812 09:40:16.449473 5973 raft_consensus.cc:2454] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 0.
I20260812 09:40:16.449520 5812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1f5eb8992f486db6ed515424f4eeb7" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5652e431dc7847b190b039d3dab856e1" is_pre_election: true
I20260812 09:40:16.449726 5812 raft_consensus.cc:2454] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 0.
I20260812 09:40:16.449905 6056 leader_election.cc:302] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9622a9a4efc04e49b5eccc2988d75ac7, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:16.450094 6279 raft_consensus.cc:2781] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:16.450157 6279 raft_consensus.cc:482] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:16.450186 6279 raft_consensus.cc:3037] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:16.451438 6279 raft_consensus.cc:504] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.451900 6279 leader_election.cc:288] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 election: Requested vote from peers 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287), d098028928d542cbafa7d0084423d64c (127.3.147.2:35119)
I20260812 09:40:16.452265 5812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1f5eb8992f486db6ed515424f4eeb7" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5652e431dc7847b190b039d3dab856e1"
I20260812 09:40:16.452425 5812 raft_consensus.cc:3037] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:16.452370 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1f5eb8992f486db6ed515424f4eeb7" candidate_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c"
I20260812 09:40:16.452546 5973 raft_consensus.cc:3037] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:16.453497 5812 raft_consensus.cc:2454] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 1.
I20260812 09:40:16.453505 5973 raft_consensus.cc:2454] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9622a9a4efc04e49b5eccc2988d75ac7 in term 1.
I20260812 09:40:16.453822 6056 leader_election.cc:302] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9622a9a4efc04e49b5eccc2988d75ac7, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:16.454059 6279 raft_consensus.cc:2781] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:16.454201 6279 raft_consensus.cc:686] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 LEADER]: Becoming Leader. State: Replica: 9622a9a4efc04e49b5eccc2988d75ac7, State: Running, Role: LEADER
I20260812 09:40:16.454370 6279 consensus_queue.cc:237] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.456182 5630 catalog_manager.cc:5495] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } health_report { overall_health: HEALTHY } } }
I20260812 09:40:16.495952 6327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:16.492403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.3.147.3:42679)
0812 09:40:16.492782 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:16.492789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:16.492809 (+ 20us) server_negotiation.cc:407] Connection header received
0812 09:40:16.492846 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:16.492849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:16.492918 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:16.493046 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:16.493351 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:16.493893 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:16.494878 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:16.495122 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:16.495303 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:16.495523 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:16.495606 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:16.495665 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:16.495743 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 09:40:16.495791 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":196,"threads_started":1}
I20260812 09:40:16.502342 5973 raft_consensus.cc:1261] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Refusing update from remote peer 9622a9a4efc04e49b5eccc2988d75ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:16.502368 5812 raft_consensus.cc:1261] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Refusing update from remote peer 9622a9a4efc04e49b5eccc2988d75ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:16.502750 6283 consensus_queue.cc:1035] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:16.502899 6279 consensus_queue.cc:1035] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:16.534740 5632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:16.534881 5632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:16.536890 5632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: Sending DeleteTablet for 3 replicas of tablet 7e1f5eb8992f486db6ed515424f4eeb7
I20260812 09:40:16.537451 5791 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1f5eb8992f486db6ed515424f4eeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:16 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35868
I20260812 09:40:16.537565 5950 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1f5eb8992f486db6ed515424f4eeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:16 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:40290
I20260812 09:40:16.537660 6108 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1f5eb8992f486db6ed515424f4eeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:16 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:52604
I20260812 09:40:16.537807 6302 tablet_replica.cc:330] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: stopping tablet replica
I20260812 09:40:16.537817 6304 tablet_replica.cc:330] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: stopping tablet replica
I20260812 09:40:16.537838 6305 tablet_replica.cc:330] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: stopping tablet replica
I20260812 09:40:16.537922 6302 raft_consensus.cc:2229] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:16.537937 6304 raft_consensus.cc:2229] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:16.537946 6305 raft_consensus.cc:2229] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:16.538007 6302 raft_consensus.cc:2258] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:16.538055 6304 raft_consensus.cc:2258] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:16.538118 6305 raft_consensus.cc:2258] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:16.538779 6305 ts_tablet_manager.cc:1905] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:16.538780 6302 ts_tablet_manager.cc:1905] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:16.539095 5632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 09:40:16.539228 6304 ts_tablet_manager.cc:1905] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 09:40:16.539455 5632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:16.541354 6305 ts_tablet_manager.cc:1918] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:16.541409 6305 log.cc:1196] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/7e1f5eb8992f486db6ed515424f4eeb7
I20260812 09:40:16.541595 6305 ts_tablet_manager.cc:1939] T 7e1f5eb8992f486db6ed515424f4eeb7 P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting consensus metadata
I20260812 09:40:16.541636 6302 ts_tablet_manager.cc:1918] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:16.541716 6302 log.cc:1196] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/7e1f5eb8992f486db6ed515424f4eeb7
I20260812 09:40:16.541826 6304 ts_tablet_manager.cc:1918] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:16.541919 6304 log.cc:1196] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/7e1f5eb8992f486db6ed515424f4eeb7
I20260812 09:40:16.542012 6302 ts_tablet_manager.cc:1939] T 7e1f5eb8992f486db6ed515424f4eeb7 P 5652e431dc7847b190b039d3dab856e1: Deleting consensus metadata
I20260812 09:40:16.542186 6304 ts_tablet_manager.cc:1939] T 7e1f5eb8992f486db6ed515424f4eeb7 P d098028928d542cbafa7d0084423d64c: Deleting consensus metadata
I20260812 09:40:16.542479 5613 catalog_manager.cc:4841] TS 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679): tablet 7e1f5eb8992f486db6ed515424f4eeb7 (table test-table [id=2eacc153aa3a4f7784be75758e1a6744]) successfully deleted
I20260812 09:40:16.542801 5615 catalog_manager.cc:4841] TS d098028928d542cbafa7d0084423d64c (127.3.147.2:35119): tablet 7e1f5eb8992f486db6ed515424f4eeb7 (table test-table [id=2eacc153aa3a4f7784be75758e1a6744]) successfully deleted
I20260812 09:40:16.543247 5612 catalog_manager.cc:4841] TS 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287): tablet 7e1f5eb8992f486db6ed515424f4eeb7 (table test-table [id=2eacc153aa3a4f7784be75758e1a6744]) successfully deleted
I20260812 09:40:16.546339 5791 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc7f488ae644e909b293cc98611204a (DEFAULT_TABLE table=test-table [id=5ec897da9c9c47f0ae8f81682d187300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:16.546485 5950 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc7f488ae644e909b293cc98611204a (DEFAULT_TABLE table=test-table [id=5ec897da9c9c47f0ae8f81682d187300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:16.546613 5791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc7f488ae644e909b293cc98611204a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:16.546571 6108 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc7f488ae644e909b293cc98611204a (DEFAULT_TABLE table=test-table [id=5ec897da9c9c47f0ae8f81682d187300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:16.546761 5950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc7f488ae644e909b293cc98611204a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:16.546798 6108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc7f488ae644e909b293cc98611204a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:16.548440 6315 tablet_bootstrap.cc:492] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: Bootstrap starting.
I20260812 09:40:16.549125 6318 tablet_bootstrap.cc:492] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1: Bootstrap starting.
I20260812 09:40:16.549296 6317 tablet_bootstrap.cc:492] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: Bootstrap starting.
I20260812 09:40:16.550104 6317 tablet_bootstrap.cc:654] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:16.550153 6318 tablet_bootstrap.cc:654] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:16.550220 6315 tablet_bootstrap.cc:654] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:16.551311 6317 tablet_bootstrap.cc:492] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: No bootstrap required, opened a new log
I20260812 09:40:16.551429 6317 ts_tablet_manager.cc:1397] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:16.551944 6317 raft_consensus.cc:348] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.552162 6317 raft_consensus.cc:374] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:16.552217 6317 raft_consensus.cc:729] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9622a9a4efc04e49b5eccc2988d75ac7, State: Initialized, Role: FOLLOWER
I20260812 09:40:16.552227 6315 tablet_bootstrap.cc:492] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: No bootstrap required, opened a new log
I20260812 09:40:16.552297 6315 ts_tablet_manager.cc:1397] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 09:40:16.552361 6317 consensus_queue.cc:260] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.552484 6318 tablet_bootstrap.cc:492] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1: No bootstrap required, opened a new log
I20260812 09:40:16.552578 6318 ts_tablet_manager.cc:1397] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260812 09:40:16.552690 6317 ts_tablet_manager.cc:1428] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:16.552774 6315 raft_consensus.cc:348] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.552899 6315 raft_consensus.cc:374] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:16.552933 6315 raft_consensus.cc:729] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d098028928d542cbafa7d0084423d64c, State: Initialized, Role: FOLLOWER
I20260812 09:40:16.553015 6318 raft_consensus.cc:348] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.553048 6315 consensus_queue.cc:260] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.553138 6318 raft_consensus.cc:374] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:16.553179 6318 raft_consensus.cc:729] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5652e431dc7847b190b039d3dab856e1, State: Initialized, Role: FOLLOWER
I20260812 09:40:16.553328 6315 ts_tablet_manager.cc:1428] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:16.553323 6318 consensus_queue.cc:260] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.553643 6318 ts_tablet_manager.cc:1428] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 09:40:16.743062 6324 raft_consensus.cc:482] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:16.743305 6324 raft_consensus.cc:504] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.743806 6324 leader_election.cc:288] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d098028928d542cbafa7d0084423d64c (127.3.147.2:35119), 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679)
I20260812 09:40:16.744297 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc7f488ae644e909b293cc98611204a" candidate_uuid: "5652e431dc7847b190b039d3dab856e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c" is_pre_election: true
I20260812 09:40:16.744323 6130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc7f488ae644e909b293cc98611204a" candidate_uuid: "5652e431dc7847b190b039d3dab856e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" is_pre_election: true
I20260812 09:40:16.744436 5973 raft_consensus.cc:2454] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5652e431dc7847b190b039d3dab856e1 in term 0.
I20260812 09:40:16.744480 6130 raft_consensus.cc:2454] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5652e431dc7847b190b039d3dab856e1 in term 0.
I20260812 09:40:16.744827 5739 leader_election.cc:302] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5652e431dc7847b190b039d3dab856e1, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:16.745028 6324 raft_consensus.cc:2781] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:16.745100 6324 raft_consensus.cc:482] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:16.745128 6324 raft_consensus.cc:3037] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:16.746213 6324 raft_consensus.cc:504] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.746565 6324 leader_election.cc:288] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [CANDIDATE]: Term 1 election: Requested vote from peers d098028928d542cbafa7d0084423d64c (127.3.147.2:35119), 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679)
I20260812 09:40:16.746953 5973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc7f488ae644e909b293cc98611204a" candidate_uuid: "5652e431dc7847b190b039d3dab856e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d098028928d542cbafa7d0084423d64c"
I20260812 09:40:16.746999 6130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc7f488ae644e909b293cc98611204a" candidate_uuid: "5652e431dc7847b190b039d3dab856e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9622a9a4efc04e49b5eccc2988d75ac7"
I20260812 09:40:16.747107 5973 raft_consensus.cc:3037] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:16.747116 6130 raft_consensus.cc:3037] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:16.748004 5973 raft_consensus.cc:2454] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5652e431dc7847b190b039d3dab856e1 in term 1.
I20260812 09:40:16.748142 6130 raft_consensus.cc:2454] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5652e431dc7847b190b039d3dab856e1 in term 1.
I20260812 09:40:16.748317 5739 leader_election.cc:302] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5652e431dc7847b190b039d3dab856e1, d098028928d542cbafa7d0084423d64c; no voters:
I20260812 09:40:16.748507 6324 raft_consensus.cc:2781] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:16.748708 6324 raft_consensus.cc:686] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [term 1 LEADER]: Becoming Leader. State: Replica: 5652e431dc7847b190b039d3dab856e1, State: Running, Role: LEADER
I20260812 09:40:16.748876 6324 consensus_queue.cc:237] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } }
I20260812 09:40:16.750560 5631 catalog_manager.cc:5495] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5652e431dc7847b190b039d3dab856e1 (127.3.147.1). New cstate: current_term: 1 leader_uuid: "5652e431dc7847b190b039d3dab856e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5652e431dc7847b190b039d3dab856e1" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 44287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 } health_report { overall_health: UNKNOWN } } }
I20260812 09:40:16.798343 6336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:16.794998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.3.147.1:44287)
0812 09:40:16.795351 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:16.795364 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:16.795376 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:40:16.795415 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:16.795417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:16.795458 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:16.795554 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:16.795820 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:16.796381 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:16.797215 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:16.797473 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:16.797587 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:16.797853 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:16.797928 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:16.798019 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:16.798113 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 09:40:16.798169 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":170,"threads_started":1}
I20260812 09:40:16.804610 5973 raft_consensus.cc:1261] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Refusing update from remote peer 5652e431dc7847b190b039d3dab856e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:16.804608 6130 raft_consensus.cc:1261] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Refusing update from remote peer 5652e431dc7847b190b039d3dab856e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:16.805025 6334 consensus_queue.cc:1035] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d098028928d542cbafa7d0084423d64c" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:16.805239 6324 consensus_queue.cc:1035] T 1dc7f488ae644e909b293cc98611204a P 5652e431dc7847b190b039d3dab856e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9622a9a4efc04e49b5eccc2988d75ac7" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 09:40:16.818506 6285 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 09:40:18.372008 5630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:18.372203 5630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:18.374660 5630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 684e3aea33c043deacc318b7f25d0b70: Sending DeleteTablet for 3 replicas of tablet 1dc7f488ae644e909b293cc98611204a
I20260812 09:40:18.375247 5950 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc7f488ae644e909b293cc98611204a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:18 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:40290
I20260812 09:40:18.375326 5791 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc7f488ae644e909b293cc98611204a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:18 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35868
I20260812 09:40:18.375401 6108 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc7f488ae644e909b293cc98611204a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:18 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:52604
I20260812 09:40:18.375880 6371 tablet_replica.cc:330] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: stopping tablet replica
I20260812 09:40:18.376116 6369 tablet_replica.cc:330] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: stopping tablet replica
I20260812 09:40:18.376231 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 5714
I20260812 09:40:18.376085 6371 raft_consensus.cc:2229] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:18.379549 6369 raft_consensus.cc:2229] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:18.379709 6369 raft_consensus.cc:2258] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:18.379693 6371 raft_consensus.cc:2258] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:18.380445 6371 ts_tablet_manager.cc:1905] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:18.380438 6369 ts_tablet_manager.cc:1905] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:18.383368 6369 ts_tablet_manager.cc:1918] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 09:40:18.383368 6371 ts_tablet_manager.cc:1918] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 09:40:18.383460 6371 log.cc:1196] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/1dc7f488ae644e909b293cc98611204a
I20260812 09:40:18.383460 6369 log.cc:1196] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/1dc7f488ae644e909b293cc98611204a
I20260812 09:40:18.383805 6371 ts_tablet_manager.cc:1939] T 1dc7f488ae644e909b293cc98611204a P 9622a9a4efc04e49b5eccc2988d75ac7: Deleting consensus metadata
I20260812 09:40:18.383801 6369 ts_tablet_manager.cc:1939] T 1dc7f488ae644e909b293cc98611204a P d098028928d542cbafa7d0084423d64c: Deleting consensus metadata
I20260812 09:40:18.384742 5615 catalog_manager.cc:4841] TS d098028928d542cbafa7d0084423d64c (127.3.147.2:35119): tablet 1dc7f488ae644e909b293cc98611204a (table test-table [id=5ec897da9c9c47f0ae8f81682d187300]) successfully deleted
I20260812 09:40:18.384824 5613 catalog_manager.cc:4841] TS 9622a9a4efc04e49b5eccc2988d75ac7 (127.3.147.3:42679): tablet 1dc7f488ae644e909b293cc98611204a (table test-table [id=5ec897da9c9c47f0ae8f81682d187300]) successfully deleted
W20260812 09:40:18.387938 5612 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:44287 (error 108)
I20260812 09:40:18.388059 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 5877
I20260812 09:40:18.393047 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:18.392607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:44287 (local address 127.0.0.1:35902)
0812 09:40:18.392890 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:18.392959 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:44287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":96,"threads_started":1}
W20260812 09:40:18.393332 5612 catalog_manager.cc:4584] TS 5652e431dc7847b190b039d3dab856e1 (127.3.147.1:44287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dc7f488ae644e909b293cc98611204a: Network error: Client connection negotiation failed: client connection to 127.3.147.1:44287: connect: Connection refused (error 111)
I20260812 09:40:18.397691 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 6036
I20260812 09:40:18.409229 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 5595
2026-08-12T09:40:18Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6770 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:18 dist-test-slave-cbbz krb5kdc[6378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:18 dist-test-slave-cbbz krb5kdc[6378](info): set up 2 sockets
Aug 12 09:40:18 dist-test-slave-cbbz krb5kdc[6378](info): commencing operation
krb5kdc: starting...
W20260812 09:40:19.515806 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:19 dist-test-slave-cbbz krb5kdc[6378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:19.637327 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38663
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:38663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:19.750365 6392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:19.750675 6392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:19.750766 6392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:19.754822 6392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:19.754920 6392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:19.754976 6392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:19.755041 6392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:19.755096 6392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:19.760234 6392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:38663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:19.761286 6392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:19.762329 6392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:19.770866 6401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:19.770902 6399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:19.770902 6403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:19.771030 6392 server_base.cc:1034] running on GCE node
I20260812 09:40:19.771739 6392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:19.772881 6392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:19.774065 6392 hybrid_clock.cc:648] HybridClock initialized: now 1786527619774056 us; error 31 us; skew 500 ppm
Aug 12 09:40:19 dist-test-slave-cbbz krb5kdc[6378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527619, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:19.781451 6392 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:19.782370 6392 webserver.cc:460] Webserver started at http://127.3.147.62:46523/ using document root <none> and password file <none>
I20260812 09:40:19.782886 6392 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:19.782943 6392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:19.783169 6392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:19.785296 6392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "053c6cc8df0447cdabe3810899ad8fe9"
format_stamp: "Formatted at 2026-08-12 09:40:19 on dist-test-slave-cbbz"
server_key: "341790c2db2e1e11911bca35b1a9400f"
server_key_iv: "372fe532cc8b7aabb242d2a721ac176e"
server_key_version: "encryptionkey@0"
I20260812 09:40:19.785871 6392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "053c6cc8df0447cdabe3810899ad8fe9"
format_stamp: "Formatted at 2026-08-12 09:40:19 on dist-test-slave-cbbz"
server_key: "341790c2db2e1e11911bca35b1a9400f"
server_key_iv: "372fe532cc8b7aabb242d2a721ac176e"
server_key_version: "encryptionkey@0"
I20260812 09:40:19.789546 6392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:19.791884 6410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:19.793226 6392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:19.793360 6392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "053c6cc8df0447cdabe3810899ad8fe9"
format_stamp: "Formatted at 2026-08-12 09:40:19 on dist-test-slave-cbbz"
server_key: "341790c2db2e1e11911bca35b1a9400f"
server_key_iv: "372fe532cc8b7aabb242d2a721ac176e"
server_key_version: "encryptionkey@0"
I20260812 09:40:19.793462 6392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:19.817060 6392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:19.819832 6392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:19.820044 6392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:19.827207 6392 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:38663
I20260812 09:40:19.827232 6472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:38663 every 8 connection(s)
I20260812 09:40:19.828163 6392 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:19.830905 6473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:19.833843 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 6392
I20260812 09:40:19.833966 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:19.834275 3660 external_mini_cluster.cc:1428] Setting key 1e3dbae8f104343bbb31e01f9b836a25
I20260812 09:40:19.837198 6473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9: Bootstrap starting.
I20260812 09:40:19.839367 6473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:19.840128 6473 log.cc:824] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:19.842175 6473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9: No bootstrap required, opened a new log
Aug 12 09:40:19 dist-test-slave-cbbz krb5kdc[6378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:19.845286 6473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053c6cc8df0447cdabe3810899ad8fe9" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38663 } }
I20260812 09:40:19.845558 6473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:19.845670 6473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053c6cc8df0447cdabe3810899ad8fe9, State: Initialized, Role: FOLLOWER
I20260812 09:40:19.846146 6473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053c6cc8df0447cdabe3810899ad8fe9" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38663 } }
I20260812 09:40:19.846299 6473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:19.846387 6473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:19.846505 6473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:19.847419 6473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053c6cc8df0447cdabe3810899ad8fe9" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38663 } }
I20260812 09:40:19.847747 6473 leader_election.cc:302] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053c6cc8df0447cdabe3810899ad8fe9; no voters:
I20260812 09:40:19.848069 6473 leader_election.cc:288] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:19.848191 6478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:19.848400 6478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [term 1 LEADER]: Becoming Leader. State: Replica: 053c6cc8df0447cdabe3810899ad8fe9, State: Running, Role: LEADER
I20260812 09:40:19.848738 6478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053c6cc8df0447cdabe3810899ad8fe9" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38663 } }
I20260812 09:40:19.849404 6473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:19.849562 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:19.835791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.3.147.62:38663)
0812 09:40:19.836312 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:19.836330 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:19.836368 (+ 38us) server_negotiation.cc:407] Connection header received
0812 09:40:19.836996 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:19.837013 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:19.837419 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:19.837788 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:19.838697 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:19.839804 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:19.840852 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:19.841363 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:19.841562 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:19.844172 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:19.844180 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:19.844202 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:19.844228 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:19.846460 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:19.846912 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:19.846918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:19.846923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:19.847042 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:19.847324 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:19.847328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:19.847330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:19.847783 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:19.847991 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:19.848324 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 09:40:19.848787 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":139,"threads_started":1}
I20260812 09:40:19.849933 6478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053c6cc8df0447cdabe3810899ad8fe9. Latest consensus state: current_term: 1 leader_uuid: "053c6cc8df0447cdabe3810899ad8fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053c6cc8df0447cdabe3810899ad8fe9" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38663 } } }
I20260812 09:40:19.850224 6478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:19.850369 6479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "053c6cc8df0447cdabe3810899ad8fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053c6cc8df0447cdabe3810899ad8fe9" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38663 } } }
I20260812 09:40:19.850539 6479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:19.851822 6488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:19.854643 6488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:19.860167 6488 catalog_manager.cc:1347] Generated new cluster ID: 821be45997b24b39a73802eab60f8fb7
I20260812 09:40:19.860229 6488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:19.881203 6488 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:19.882218 6488 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:19.887291 6488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 053c6cc8df0447cdabe3810899ad8fe9: Generated new TSK 0
I20260812 09:40:19.888104 6488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:19.919160 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38663
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:20.032245 6500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:20.032480 6500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:20.032522 6500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:20.036633 6500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:20.036681 6500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:20.036774 6500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:20.042028 6500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:20.043017 6500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:20.044049 6500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:40:20.054418 6500 server_base.cc:1034] running on GCE node
W20260812 09:40:20.054349 6506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:20.054487 6505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:20.054368 6508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:20.054971 6500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:20.055579 6500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:20.056736 6500 hybrid_clock.cc:648] HybridClock initialized: now 1786527620056701 us; error 48 us; skew 500 ppm
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527620, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:20.059845 6500 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:20.060770 6500 webserver.cc:460] Webserver started at http://127.3.147.1:38725/ using document root <none> and password file <none>
I20260812 09:40:20.061319 6500 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:20.061410 6500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:20.061662 6500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:20.063673 6500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "2f3a920b2dc84ff2bf102fcd5309e9b4"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "c08b40d6c069750c4c97a17e4cfcfd3c"
server_key_iv: "9e7d2c94fa8ff755b57ea47231fb49fc"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.064296 6500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "2f3a920b2dc84ff2bf102fcd5309e9b4"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "c08b40d6c069750c4c97a17e4cfcfd3c"
server_key_iv: "9e7d2c94fa8ff755b57ea47231fb49fc"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.067792 6500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:20.070128 6517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.071269 6500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:20.071424 6500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "2f3a920b2dc84ff2bf102fcd5309e9b4"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "c08b40d6c069750c4c97a17e4cfcfd3c"
server_key_iv: "9e7d2c94fa8ff755b57ea47231fb49fc"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.071530 6500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:20.085837 6500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:20.089051 6500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:20.089251 6500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:20.089825 6500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:20.090656 6500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:20.090725 6500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.090794 6500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:20.090849 6500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.100513 6500 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:45743
I20260812 09:40:20.100554 6663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:45743 every 8 connection(s)
I20260812 09:40:20.101375 6500 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:20.105228 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 6500
I20260812 09:40:20.105324 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:20.105664 3660 external_mini_cluster.cc:1428] Setting key eaa16afcea435f2666bd8b5466d6d716
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527620, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:20.116247 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:20.103137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:51629 (local address 127.3.147.62:38663)
0812 09:40:20.103288 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:20.103292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:20.104072 (+ 780us) server_negotiation.cc:407] Connection header received
0812 09:40:20.105514 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:20.105519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:20.105580 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:20.105678 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:20.106948 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.107715 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.108939 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.109169 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.109281 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:20.112239 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:20.112242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:20.112244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:20.112267 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:20.113776 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:20.114391 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:20.114394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:20.114395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:20.114468 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:20.114925 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:20.114928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:20.114929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:20.115144 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:20.115258 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:20.115921 (+ 663us) server_negotiation.cc:299] Negotiation successful
0812 09:40:20.116035 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:20.117089 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:20.103489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38663 (local address 127.3.147.1:51629)
0812 09:40:20.103917 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:20.103958 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:20.105199 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:20.105869 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:20.105875 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:20.106227 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:20.106796 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:20.106813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.107857 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.107859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:20.108808 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:20.108817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.109315 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.109319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:20.109483 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:20.110067 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:20.110082 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:20.112027 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:20.113903 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:20.113909 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:20.113922 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:20.114286 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:20.114558 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:20.114561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:20.114564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:20.114760 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:20.115289 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:20.115295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:20.115604 (+ 309us) client_negotiation.cc:769] Sending connection context
0812 09:40:20.115833 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 09:40:20.116190 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":111,"threads_started":1}
I20260812 09:40:20.118240 6664 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38663
I20260812 09:40:20.118501 6664 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:20.118976 6664 heartbeater.cc:511] Master 127.3.147.62:38663 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:40:20.120705 6431 ts_manager.cc:194] Registered new tserver with Master: 2f3a920b2dc84ff2bf102fcd5309e9b4 (127.3.147.1:45743)
I20260812 09:40:20.121843 6431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:51629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:20.135382 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38663
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:20.248490 6670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:20.248749 6670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:20.248792 6670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:20.253180 6670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:20.253237 6670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:20.253294 6670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:20.258510 6670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:20.259474 6670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:20.260524 6670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:20.270376 6681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:20.270376 6678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:20.270529 6670 server_base.cc:1034] running on GCE node
W20260812 09:40:20.270725 6679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:20.271200 6670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:20.271865 6670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:20.273051 6670 hybrid_clock.cc:648] HybridClock initialized: now 1786527620273018 us; error 45 us; skew 500 ppm
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527620, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:20.275812 6670 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:20.276664 6670 webserver.cc:460] Webserver started at http://127.3.147.2:33415/ using document root <none> and password file <none>
I20260812 09:40:20.277202 6670 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:20.277259 6670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:20.277441 6670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:20.279418 6670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "695a14a2cb1d487cafaab20a634a0409"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "2d07c3736e1f286ed51a1d1806edd880"
server_key_iv: "b3bf63b7667ce8dd579e858269976362"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.279927 6670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "695a14a2cb1d487cafaab20a634a0409"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "2d07c3736e1f286ed51a1d1806edd880"
server_key_iv: "b3bf63b7667ce8dd579e858269976362"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.283584 6670 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:20.286002 6688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.287242 6670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 09:40:20.287338 6670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "695a14a2cb1d487cafaab20a634a0409"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "2d07c3736e1f286ed51a1d1806edd880"
server_key_iv: "b3bf63b7667ce8dd579e858269976362"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.287421 6670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:20.326958 6670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:20.330462 6670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:20.330619 6670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:20.331190 6670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:20.331977 6670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:20.332024 6670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.332064 6670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:20.332079 6670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.341495 6670 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:34963
I20260812 09:40:20.341521 6821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:34963 every 8 connection(s)
I20260812 09:40:20.342446 6670 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:20.352111 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 6670
I20260812 09:40:20.352241 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:20.352550 3660 external_mini_cluster.cc:1428] Setting key 072de95944350244ff3037322cc7f2aa
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527620, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:20.357239 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:20.344297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:45873 (local address 127.3.147.62:38663)
0812 09:40:20.344440 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:20.344444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:20.345344 (+ 900us) server_negotiation.cc:407] Connection header received
0812 09:40:20.346611 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:20.346614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:20.346658 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:20.346734 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:20.347857 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.348440 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.349719 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.350040 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.350247 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:20.353432 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:20.353439 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:20.353443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:20.353498 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:20.354986 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:20.355522 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:20.355527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:20.355529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:20.355622 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:20.356023 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:20.356029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:20.356032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:20.356318 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:20.356402 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:20.357022 (+ 620us) server_negotiation.cc:299] Negotiation successful
0812 09:40:20.357123 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 09:40:20.358441 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:20.344694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38663 (local address 127.3.147.2:45873)
0812 09:40:20.345182 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:20.345224 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:20.346411 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:20.346900 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:20.346906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:20.347267 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:20.347722 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:20.347734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.348576 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.348579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:20.349612 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:20.349622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.350221 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.350228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:20.350460 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:20.351188 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:20.351206 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:20.353244 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:20.355127 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:20.355136 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:20.355150 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:20.355414 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:20.355717 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:20.355721 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:20.355723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:20.355879 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:20.356428 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:20.356437 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:20.356765 (+ 328us) client_negotiation.cc:769] Sending connection context
0812 09:40:20.356979 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 09:40:20.357328 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":132,"threads_started":1}
I20260812 09:40:20.359676 6822 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38663
I20260812 09:40:20.359915 6822 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:20.360447 6822 heartbeater.cc:511] Master 127.3.147.62:38663 requested a full tablet report, sending...
I20260812 09:40:20.361502 6431 ts_manager.cc:194] Registered new tserver with Master: 695a14a2cb1d487cafaab20a634a0409 (127.3.147.2:34963)
I20260812 09:40:20.362071 6431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:45873
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:20.382854 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38663
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:20.495038 6829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:20.495322 6829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:20.495395 6829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:20.499603 6829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:20.499686 6829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:20.499774 6829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:20.504930 6829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:20.505990 6829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:20.507028 6829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:20.516666 6835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:20.516752 6834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:20.516867 6829 server_base.cc:1034] running on GCE node
W20260812 09:40:20.516665 6838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:20.517369 6829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:20.518033 6829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:20.519208 6829 hybrid_clock.cc:648] HybridClock initialized: now 1786527620519191 us; error 29 us; skew 500 ppm
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527620, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:20.521978 6829 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:20.522799 6829 webserver.cc:460] Webserver started at http://127.3.147.3:37121/ using document root <none> and password file <none>
I20260812 09:40:20.523350 6829 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:20.523438 6829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:20.523667 6829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:20.525743 6829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "a439573e5ac64b30a2a3daaac753d10b"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "76a66d6f71c9fc38cab59880cf1808d8"
server_key_iv: "c47d2ba4c1d17b9ca50876d09598ce8c"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.526309 6829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "a439573e5ac64b30a2a3daaac753d10b"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "76a66d6f71c9fc38cab59880cf1808d8"
server_key_iv: "c47d2ba4c1d17b9ca50876d09598ce8c"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.530057 6829 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:40:20.532428 6846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.533612 6829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:20.533759 6829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "a439573e5ac64b30a2a3daaac753d10b"
format_stamp: "Formatted at 2026-08-12 09:40:20 on dist-test-slave-cbbz"
server_key: "76a66d6f71c9fc38cab59880cf1808d8"
server_key_iv: "c47d2ba4c1d17b9ca50876d09598ce8c"
server_key_version: "encryptionkey@0"
I20260812 09:40:20.533874 6829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:20.566857 6829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:20.570531 6829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:20.570746 6829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:20.571390 6829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:20.572315 6829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:20.572382 6829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.572453 6829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:20.572492 6829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:20.582217 6829 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:38651
I20260812 09:40:20.582237 6984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:38651 every 8 connection(s)
I20260812 09:40:20.583141 6829 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:20.589140 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 6829
I20260812 09:40:20.589255 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:20.589558 3660 external_mini_cluster.cc:1428] Setting key 5c8c47455be3d612e09fb2aae53222f2
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527620, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:20.597289 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:20.585083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:51427 (local address 127.3.147.62:38663)
0812 09:40:20.585242 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:20.585247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:20.586097 (+ 850us) server_negotiation.cc:407] Connection header received
0812 09:40:20.587392 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:20.587395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:20.587443 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:20.587519 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:20.588758 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.589592 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.591177 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.591478 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.591592 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:20.594201 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:20.594204 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:20.594206 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:20.594231 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:20.595404 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:20.595866 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:20.595869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:20.595870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:20.595942 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:20.596320 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:20.596322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:20.596323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:20.596489 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:20.596565 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:20.597064 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 09:40:20.597167 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 09:40:20.598191 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:20.585498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38663 (local address 127.3.147.3:51427)
0812 09:40:20.585949 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:20.585992 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:20.587204 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:20.587696 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:20.587704 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:20.588066 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:20.588610 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:20.588624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.589731 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.589734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:20.591057 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:20.591067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:20.591653 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:20.591661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:20.591876 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:20.592501 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:20.592517 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:20.594049 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:20.595510 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:20.595515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:20.595526 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:20.595782 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:20.596018 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:20.596020 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:20.596022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:20.596238 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:20.596598 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:20.596605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:20.596834 (+ 229us) client_negotiation.cc:769] Sending connection context
0812 09:40:20.597012 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 09:40:20.597279 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260812 09:40:20.599346 6985 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38663
I20260812 09:40:20.599596 6985 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:20.600096 6985 heartbeater.cc:511] Master 127.3.147.62:38663 requested a full tablet report, sending...
I20260812 09:40:20.601127 6431 ts_manager.cc:194] Registered new tserver with Master: a439573e5ac64b30a2a3daaac753d10b (127.3.147.3:38651)
I20260812 09:40:20.601701 6431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:51427
I20260812 09:40:20.603613 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 09:40:20.617779 6476 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:20.616403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.3.147.62:38663)
0812 09:40:20.616540 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:20.616544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:20.616715 (+ 171us) server_negotiation.cc:407] Connection header received
0812 09:40:20.616805 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:20.616807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:20.616843 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:20.616911 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:20.617644 (+ 733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60442: BlockingRecv error: recv got EOF from 127.0.0.1:60442 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 09:40:20.618765 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 6500
I20260812 09:40:20.632694 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 6670
I20260812 09:40:20.643999 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 6829
I20260812 09:40:20.659309 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 6392
2026-08-12T09:40:20Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2251 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6998](info): set up 2 sockets
Aug 12 09:40:20 dist-test-slave-cbbz krb5kdc[6998](info): commencing operation
krb5kdc: starting...
W20260812 09:40:21.766582 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:21 dist-test-slave-cbbz krb5kdc[6998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:21.887109 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41555
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:41555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:21.996776 7012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:21.997007 7012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:21.997046 7012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:22.000969 7012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:22.001016 7012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:22.001036 7012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:22.001052 7012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:22.001101 7012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:22.006771 7012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:41555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:22.007754 7012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:22.008793 7012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:22.016927 7020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:22.016942 7017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:22.016971 7018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:22.017249 7012 server_base.cc:1034] running on GCE node
I20260812 09:40:22.017802 7012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:22.018826 7012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:22.020013 7012 hybrid_clock.cc:648] HybridClock initialized: now 1786527622019996 us; error 30 us; skew 500 ppm
Aug 12 09:40:22 dist-test-slave-cbbz krb5kdc[6998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527622, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:22.022740 7012 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:22.023505 7012 webserver.cc:460] Webserver started at http://127.3.147.62:42823/ using document root <none> and password file <none>
I20260812 09:40:22.023998 7012 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:22.024055 7012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:22.024284 7012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:22.026189 7012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "dc61a169d50c4fe993a25c03d497fe80"
format_stamp: "Formatted at 2026-08-12 09:40:22 on dist-test-slave-cbbz"
server_key: "cf6cfaf1ef21b305528e2befc2465657"
server_key_iv: "4fb41e56277fb054be6ff3de9ccc2ebc"
server_key_version: "encryptionkey@0"
I20260812 09:40:22.026736 7012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "dc61a169d50c4fe993a25c03d497fe80"
format_stamp: "Formatted at 2026-08-12 09:40:22 on dist-test-slave-cbbz"
server_key: "cf6cfaf1ef21b305528e2befc2465657"
server_key_iv: "4fb41e56277fb054be6ff3de9ccc2ebc"
server_key_version: "encryptionkey@0"
I20260812 09:40:22.030218 7012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:40:22.032423 7027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:22.033571 7012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:40:22.033712 7012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "dc61a169d50c4fe993a25c03d497fe80"
format_stamp: "Formatted at 2026-08-12 09:40:22 on dist-test-slave-cbbz"
server_key: "cf6cfaf1ef21b305528e2befc2465657"
server_key_iv: "4fb41e56277fb054be6ff3de9ccc2ebc"
server_key_version: "encryptionkey@0"
I20260812 09:40:22.033805 7012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:22.053694 7012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:22.056478 7012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:22.056672 7012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:22.063796 7012 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:41555
I20260812 09:40:22.064226 7090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:41555 every 8 connection(s)
I20260812 09:40:22.064826 7012 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:22.067485 7091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:22.072909 7091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80: Bootstrap starting.
I20260812 09:40:22.073277 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 7012
I20260812 09:40:22.073429 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:22.073695 3660 external_mini_cluster.cc:1428] Setting key e546d0dbc50b992f78a401c5e86c7c7d
I20260812 09:40:22.075593 7091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:22.076493 7091 log.cc:824] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:22.078629 7091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80: No bootstrap required, opened a new log
I20260812 09:40:22.082588 7091 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc61a169d50c4fe993a25c03d497fe80" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41555 } }
I20260812 09:40:22.082794 7091 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:22.082868 7091 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc61a169d50c4fe993a25c03d497fe80, State: Initialized, Role: FOLLOWER
Aug 12 09:40:22 dist-test-slave-cbbz krb5kdc[6998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:22.083376 7091 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc61a169d50c4fe993a25c03d497fe80" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41555 } }
I20260812 09:40:22.083549 7091 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:22.083621 7091 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:22.083727 7091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:22.084666 7091 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc61a169d50c4fe993a25c03d497fe80" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41555 } }
I20260812 09:40:22.085021 7091 leader_election.cc:302] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc61a169d50c4fe993a25c03d497fe80; no voters:
I20260812 09:40:22.085335 7091 leader_election.cc:288] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:22.085502 7096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:22.085713 7096 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [term 1 LEADER]: Becoming Leader. State: Replica: dc61a169d50c4fe993a25c03d497fe80, State: Running, Role: LEADER
I20260812 09:40:22.085994 7096 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc61a169d50c4fe993a25c03d497fe80" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41555 } }
I20260812 09:40:22.086611 7091 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:22.087143 7096 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc61a169d50c4fe993a25c03d497fe80. Latest consensus state: current_term: 1 leader_uuid: "dc61a169d50c4fe993a25c03d497fe80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc61a169d50c4fe993a25c03d497fe80" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41555 } } }
I20260812 09:40:22.087255 7096 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:22.087553 7098 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc61a169d50c4fe993a25c03d497fe80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc61a169d50c4fe993a25c03d497fe80" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41555 } } }
I20260812 09:40:22.087679 7103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:22.087690 7098 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:22.089956 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:22.074980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49488 (local address 127.3.147.62:41555)
0812 09:40:22.075473 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:22.075494 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:22.075558 (+ 64us) server_negotiation.cc:407] Connection header received
0812 09:40:22.076432 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:22.076455 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:22.076864 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:22.077146 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:22.077942 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:22.079364 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:22.080287 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:22.080974 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:22.081151 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:22.083809 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:22.083818 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:22.083836 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:22.083868 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:22.085865 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:22.086391 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:22.086397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:22.086403 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:22.086523 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:22.086806 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:22.086810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:22.086812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:22.087482 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:22.088396 (+ 914us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:22.088555 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 09:40:22.088887 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":113,"threads_started":1}
I20260812 09:40:22.091221 7103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:22.093246 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:40:22.097223 7103 catalog_manager.cc:1347] Generated new cluster ID: 3f77af29b61c465283f594f1982aa7bc
I20260812 09:40:22.097297 7103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:22.121274 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:22.115550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.3.147.62:41555)
0812 09:40:22.115668 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:22.115672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:22.115935 (+ 263us) server_negotiation.cc:407] Connection header received
0812 09:40:22.116050 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:22.116053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:22.116109 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:22.116220 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:22.116755 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:22.117559 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:22.118680 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:22.119089 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:22.119212 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:22.120188 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:22.120193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:22.120196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:22.120787 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:22.120820 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:22.120839 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:22.120932 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:22.121065 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 09:40:22.121124 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 09:40:22.121773 7103 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:22.122830 7103 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:22.129840 7103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc61a169d50c4fe993a25c03d497fe80: Generated new TSK 0
I20260812 09:40:22.130576 7103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:40:22.142844 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 7012
2026-08-12T09:40:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1482 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:22 dist-test-slave-cbbz krb5kdc[7129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:22 dist-test-slave-cbbz krb5kdc[7129](info): set up 2 sockets
Aug 12 09:40:22 dist-test-slave-cbbz krb5kdc[7129](info): commencing operation
krb5kdc: starting...
W20260812 09:40:23.269985 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:23 dist-test-slave-cbbz krb5kdc[7129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:23.399467 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:46243
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:23.510453 7143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:23.510787 7143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:23.510869 7143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:23.514859 7143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:23.514945 7143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:23.514984 7143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:23.515017 7143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:23.515048 7143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:23.520399 7143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:46243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:23.521413 7143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:23.522423 7143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:23.530740 7149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:23.530748 7152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:23.530748 7148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:23.530912 7143 server_base.cc:1034] running on GCE node
I20260812 09:40:23.531564 7143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:23.532622 7143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:23.533785 7143 hybrid_clock.cc:648] HybridClock initialized: now 1786527623533750 us; error 46 us; skew 500 ppm
Aug 12 09:40:23 dist-test-slave-cbbz krb5kdc[7129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527623, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:23.536471 7143 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:23.537360 7143 webserver.cc:460] Webserver started at http://127.3.147.62:36979/ using document root <none> and password file <none>
I20260812 09:40:23.537894 7143 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:23.537957 7143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:23.538211 7143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:23.540261 7143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "3332939d4493433f888a96a3f5535b6f"
format_stamp: "Formatted at 2026-08-12 09:40:23 on dist-test-slave-cbbz"
server_key: "7efe93f8b3cecd1c058f2fea244bee24"
server_key_iv: "eacd036ce1bee965e1aad5bb423e61f4"
server_key_version: "encryptionkey@0"
I20260812 09:40:23.540819 7143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "3332939d4493433f888a96a3f5535b6f"
format_stamp: "Formatted at 2026-08-12 09:40:23 on dist-test-slave-cbbz"
server_key: "7efe93f8b3cecd1c058f2fea244bee24"
server_key_iv: "eacd036ce1bee965e1aad5bb423e61f4"
server_key_version: "encryptionkey@0"
I20260812 09:40:23.544449 7143 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:40:23.546751 7160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:23.547948 7143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:23.548097 7143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "3332939d4493433f888a96a3f5535b6f"
format_stamp: "Formatted at 2026-08-12 09:40:23 on dist-test-slave-cbbz"
server_key: "7efe93f8b3cecd1c058f2fea244bee24"
server_key_iv: "eacd036ce1bee965e1aad5bb423e61f4"
server_key_version: "encryptionkey@0"
I20260812 09:40:23.548288 7143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:23.559001 7143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:23.561690 7143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:23.561906 7143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:23.569154 7143 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:46243
I20260812 09:40:23.569438 7218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:46243 every 8 connection(s)
I20260812 09:40:23.570134 7143 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:23.572849 7219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:23.576251 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 7143
I20260812 09:40:23.576378 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:23.576678 3660 external_mini_cluster.cc:1428] Setting key 54d4b9d299e4e7362fa505c00e61c40e
I20260812 09:40:23.578879 7219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f: Bootstrap starting.
I20260812 09:40:23.581424 7219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:23.582291 7219 log.cc:824] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:23.584542 7219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f: No bootstrap required, opened a new log
Aug 12 09:40:23 dist-test-slave-cbbz krb5kdc[7129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:23.588124 7219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3332939d4493433f888a96a3f5535b6f" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46243 } }
I20260812 09:40:23.588423 7219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:23.588510 7219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3332939d4493433f888a96a3f5535b6f, State: Initialized, Role: FOLLOWER
I20260812 09:40:23.588963 7219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3332939d4493433f888a96a3f5535b6f" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46243 } }
I20260812 09:40:23.589116 7219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:23.589190 7219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:23.589322 7219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:23.590235 7219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3332939d4493433f888a96a3f5535b6f" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46243 } }
I20260812 09:40:23.590593 7219 leader_election.cc:302] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3332939d4493433f888a96a3f5535b6f; no voters:
I20260812 09:40:23.590927 7219 leader_election.cc:288] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:23.591002 7227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:23.591197 7227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [term 1 LEADER]: Becoming Leader. State: Replica: 3332939d4493433f888a96a3f5535b6f, State: Running, Role: LEADER
I20260812 09:40:23.591559 7227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3332939d4493433f888a96a3f5535b6f" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46243 } }
I20260812 09:40:23.591766 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:23.578094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.3.147.62:46243)
0812 09:40:23.578517 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:23.578541 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:23.578577 (+ 36us) server_negotiation.cc:407] Connection header received
0812 09:40:23.579263 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:23.579282 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:23.579614 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:23.579890 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:23.580809 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:23.581855 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:23.582989 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:23.583484 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:23.583601 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:23.586462 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:23.586467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:23.586480 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:23.586507 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:23.588312 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:23.588879 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:23.588884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:23.588888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:23.588998 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:23.589328 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:23.589332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:23.589334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:23.589772 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:23.589982 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:23.590309 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 09:40:23.590768 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":124,"threads_started":1}
I20260812 09:40:23.592291 7219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:23.593338 7230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3332939d4493433f888a96a3f5535b6f. Latest consensus state: current_term: 1 leader_uuid: "3332939d4493433f888a96a3f5535b6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3332939d4493433f888a96a3f5535b6f" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46243 } } }
I20260812 09:40:23.593462 7230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:23.593875 7229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3332939d4493433f888a96a3f5535b6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3332939d4493433f888a96a3f5535b6f" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46243 } } }
I20260812 09:40:23.594493 7229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:23.594760 7237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:23.597443 7237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:23.602934 7237 catalog_manager.cc:1347] Generated new cluster ID: 3f1d7ce6b461497c89aff0edcc2429ff
I20260812 09:40:23.602998 7237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:23.628216 7237 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:23.629572 7237 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:23.634284 7237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3332939d4493433f888a96a3f5535b6f: Generated new TSK 0
I20260812 09:40:23.635059 7237 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:23.679754 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:46243
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:23.800365 7251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:23.800619 7251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:23.800663 7251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:23.804677 7251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:23.804733 7251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:23.804831 7251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:23.810128 7251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:23.811187 7251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:23.812317 7251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:23.822108 7257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:23.822166 7258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:23.822284 7251 server_base.cc:1034] running on GCE node
W20260812 09:40:23.822110 7262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:23.822840 7251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:23.823400 7251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:23.824559 7251 hybrid_clock.cc:648] HybridClock initialized: now 1786527623824539 us; error 30 us; skew 500 ppm
Aug 12 09:40:23 dist-test-slave-cbbz krb5kdc[7129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527623, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:23.827380 7251 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:23.828305 7251 webserver.cc:460] Webserver started at http://127.3.147.1:45201/ using document root <none> and password file <none>
I20260812 09:40:23.828886 7251 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:23.828949 7251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:23.829191 7251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:23.831188 7251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "2f3855d0b8ef4da9a857c6e166275160"
format_stamp: "Formatted at 2026-08-12 09:40:23 on dist-test-slave-cbbz"
server_key: "8113f3ec59b804fa02685014b52f7b71"
server_key_iv: "7e98e375e804a51ee8333859408fa357"
server_key_version: "encryptionkey@0"
I20260812 09:40:23.831756 7251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "2f3855d0b8ef4da9a857c6e166275160"
format_stamp: "Formatted at 2026-08-12 09:40:23 on dist-test-slave-cbbz"
server_key: "8113f3ec59b804fa02685014b52f7b71"
server_key_iv: "7e98e375e804a51ee8333859408fa357"
server_key_version: "encryptionkey@0"
I20260812 09:40:23.835338 7251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 09:40:23.837705 7271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:23.838893 7251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:23.839026 7251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "2f3855d0b8ef4da9a857c6e166275160"
format_stamp: "Formatted at 2026-08-12 09:40:23 on dist-test-slave-cbbz"
server_key: "8113f3ec59b804fa02685014b52f7b71"
server_key_iv: "7e98e375e804a51ee8333859408fa357"
server_key_version: "encryptionkey@0"
I20260812 09:40:23.839131 7251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:23.849185 7251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:23.851956 7251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:23.852252 7251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:23.852818 7251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:23.853587 7251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:23.853665 7251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:23.853730 7251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:23.853780 7251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:23.863322 7251 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:45859
I20260812 09:40:23.863358 7398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:45859 every 8 connection(s)
I20260812 09:40:23.864240 7251 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:23.866834 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 7251
I20260812 09:40:23.866971 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:23.867369 3660 external_mini_cluster.cc:1428] Setting key ab39d9c673922ed028427a3e9f05515b
Aug 12 09:40:23 dist-test-slave-cbbz krb5kdc[7129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527623, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:23.879491 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:23.866096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:58903 (local address 127.3.147.62:46243)
0812 09:40:23.866289 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:23.866295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:23.867232 (+ 937us) server_negotiation.cc:407] Connection header received
0812 09:40:23.869008 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:23.869014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:23.869070 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:23.869166 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:23.870299 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:23.871128 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:23.872896 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:23.873113 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:23.873193 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:23.875894 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:23.875903 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:23.875905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:23.875928 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:23.877400 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:23.877951 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:23.877959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:23.877961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:23.878036 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:23.878398 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:23.878401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:23.878403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:23.878591 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:23.878705 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:23.879191 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 09:40:23.879321 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 09:40:23.880350 7401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:23.866512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:46243 (local address 127.3.147.1:58903)
0812 09:40:23.867047 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:23.867109 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:23.868724 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:23.869321 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:23.869327 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:23.869690 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:23.870143 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:23.870156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:23.871270 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:23.871276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:23.872777 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:23.872791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:23.873282 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:23.873288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:23.873523 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:23.874244 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:23.874260 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:23.875714 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:23.877535 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:23.877540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:23.877554 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:23.877848 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:23.878137 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:23.878141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:23.878142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:23.878289 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:23.878706 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:23.878713 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:23.878958 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 09:40:23.879151 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 09:40:23.879483 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:40:23.881716 7399 heartbeater.cc:347] Connected to a master server at 127.3.147.62:46243
I20260812 09:40:23.882005 7399 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:23.882498 7399 heartbeater.cc:511] Master 127.3.147.62:46243 requested a full tablet report, sending...
I20260812 09:40:23.884004 7178 ts_manager.cc:194] Registered new tserver with Master: 2f3855d0b8ef4da9a857c6e166275160 (127.3.147.1:45859)
I20260812 09:40:23.885178 7178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:58903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:23.898891 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:46243
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:24.010591 7404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:24.010851 7404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:24.010895 7404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:24.014870 7404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:24.014923 7404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:24.014981 7404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:24.020334 7404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:24.021322 7404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:24.022292 7404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:24.031862 7410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:24.031912 7409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:24.031919 7412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:24.032192 7404 server_base.cc:1034] running on GCE node
I20260812 09:40:24.032680 7404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:24.033300 7404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:24.034494 7404 hybrid_clock.cc:648] HybridClock initialized: now 1786527624034466 us; error 46 us; skew 500 ppm
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527624, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:24.037395 7404 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:24.038216 7404 webserver.cc:460] Webserver started at http://127.3.147.2:35409/ using document root <none> and password file <none>
I20260812 09:40:24.038781 7404 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:24.038874 7404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:24.039103 7404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:24.041169 7404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "fc2e0d66dbb046deb9bbd665db80a91b"
format_stamp: "Formatted at 2026-08-12 09:40:24 on dist-test-slave-cbbz"
server_key: "5afcbb05c80e3c25fe8ee8e292f481ce"
server_key_iv: "fc6dffbba6be5a76b1931cbc51aea1d0"
server_key_version: "encryptionkey@0"
I20260812 09:40:24.041764 7404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "fc2e0d66dbb046deb9bbd665db80a91b"
format_stamp: "Formatted at 2026-08-12 09:40:24 on dist-test-slave-cbbz"
server_key: "5afcbb05c80e3c25fe8ee8e292f481ce"
server_key_iv: "fc6dffbba6be5a76b1931cbc51aea1d0"
server_key_version: "encryptionkey@0"
I20260812 09:40:24.045526 7404 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:24.047890 7420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:24.049086 7404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:40:24.049223 7404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "fc2e0d66dbb046deb9bbd665db80a91b"
format_stamp: "Formatted at 2026-08-12 09:40:24 on dist-test-slave-cbbz"
server_key: "5afcbb05c80e3c25fe8ee8e292f481ce"
server_key_iv: "fc6dffbba6be5a76b1931cbc51aea1d0"
server_key_version: "encryptionkey@0"
I20260812 09:40:24.049336 7404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:24.070026 7404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:24.072889 7404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:24.073100 7404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:24.073753 7404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:24.074615 7404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:24.074685 7404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:24.074751 7404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:24.074793 7404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:24.084163 7404 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:45397
I20260812 09:40:24.084261 7551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:45397 every 8 connection(s)
I20260812 09:40:24.085151 7404 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:24.085717 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 7404
I20260812 09:40:24.085852 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:24.086172 3660 external_mini_cluster.cc:1428] Setting key 70d6912fe224160fd4a4c2c8b8deabe4
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527624, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:40:24.100883 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:24.087053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:36385 (local address 127.3.147.62:46243)
0812 09:40:24.087199 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:24.087203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:24.088257 (+ 1054us) server_negotiation.cc:407] Connection header received
0812 09:40:24.089743 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:24.089746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:24.089799 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:24.089869 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:24.091445 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.092379 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.093777 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.094193 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.094360 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:24.097153 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:24.097161 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:24.097167 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:24.097201 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:24.098635 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:24.099127 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:24.099130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:24.099131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:24.099201 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:24.099636 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:24.099639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:24.099640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:24.099825 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:24.099904 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:24.100552 (+ 648us) server_negotiation.cc:299] Negotiation successful
0812 09:40:24.100694 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:40:24.101688 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:24.087517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:46243 (local address 127.3.147.2:36385)
0812 09:40:24.088070 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:24.088113 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:24.089456 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:24.090054 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:24.090064 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:24.090642 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:24.091295 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:24.091312 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.092504 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.092507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:24.093618 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:24.093628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.094393 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.094397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:24.094530 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:24.095319 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:24.095336 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:24.096974 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:24.098751 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:24.098756 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:24.098768 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:24.099039 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:24.099295 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:24.099298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:24.099300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:24.099541 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:24.099948 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:24.099954 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:24.100302 (+ 348us) client_negotiation.cc:769] Sending connection context
0812 09:40:24.100503 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 09:40:24.100831 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":204,"threads_started":1}
I20260812 09:40:24.102977 7552 heartbeater.cc:347] Connected to a master server at 127.3.147.62:46243
I20260812 09:40:24.103222 7552 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:24.103709 7552 heartbeater.cc:511] Master 127.3.147.62:46243 requested a full tablet report, sending...
I20260812 09:40:24.104817 7178 ts_manager.cc:194] Registered new tserver with Master: fc2e0d66dbb046deb9bbd665db80a91b (127.3.147.2:45397)
I20260812 09:40:24.105391 7178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:36385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:24.115311 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:46243
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:24.226604 7557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:24.226840 7557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:24.226882 7557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:24.230945 7557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:24.230999 7557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:24.231055 7557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:24.236100 7557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:24.237074 7557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:24.238018 7557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:24.247695 7564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:24.247695 7565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:24.247716 7567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:24.248015 7557 server_base.cc:1034] running on GCE node
I20260812 09:40:24.248539 7557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:24.249114 7557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:24.250279 7557 hybrid_clock.cc:648] HybridClock initialized: now 1786527624250261 us; error 30 us; skew 500 ppm
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527624, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:24.253278 7557 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:24.254315 7557 webserver.cc:460] Webserver started at http://127.3.147.3:41661/ using document root <none> and password file <none>
I20260812 09:40:24.254878 7557 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:24.254973 7557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:24.255208 7557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:24.257299 7557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "df0e66dbe88249b9bcaadea327cf34ff"
format_stamp: "Formatted at 2026-08-12 09:40:24 on dist-test-slave-cbbz"
server_key: "2de8711f8c3f6c2bb6ae34b5ec7ced00"
server_key_iv: "621bf5efb049eb4c6e6a013a785d413f"
server_key_version: "encryptionkey@0"
I20260812 09:40:24.257870 7557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "df0e66dbe88249b9bcaadea327cf34ff"
format_stamp: "Formatted at 2026-08-12 09:40:24 on dist-test-slave-cbbz"
server_key: "2de8711f8c3f6c2bb6ae34b5ec7ced00"
server_key_iv: "621bf5efb049eb4c6e6a013a785d413f"
server_key_version: "encryptionkey@0"
I20260812 09:40:24.261571 7557 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:24.263948 7575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:24.265203 7557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:24.265352 7557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "df0e66dbe88249b9bcaadea327cf34ff"
format_stamp: "Formatted at 2026-08-12 09:40:24 on dist-test-slave-cbbz"
server_key: "2de8711f8c3f6c2bb6ae34b5ec7ced00"
server_key_iv: "621bf5efb049eb4c6e6a013a785d413f"
server_key_version: "encryptionkey@0"
I20260812 09:40:24.265465 7557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:24.289708 7557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:24.293370 7557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:24.293602 7557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:24.294320 7557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:24.295176 7557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:24.295249 7557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:24.295316 7557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:24.295365 7557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:24.304399 7557 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:40621
I20260812 09:40:24.304471 7709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:40621 every 8 connection(s)
I20260812 09:40:24.305315 7557 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:24.311589 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 7557
I20260812 09:40:24.311715 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:24.312017 3660 external_mini_cluster.cc:1428] Setting key 07c25b35a61546019c841e9fc656c72a
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527624, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:24.318557 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:24.307116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:43003 (local address 127.3.147.62:46243)
0812 09:40:24.307302 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:24.307307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:24.308165 (+ 858us) server_negotiation.cc:407] Connection header received
0812 09:40:24.309544 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:24.309548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:24.309592 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:24.309666 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:24.310761 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.311280 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.312744 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.313008 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.313116 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:24.315602 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:24.315605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:24.315607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:24.315629 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:24.316882 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:24.317316 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:24.317319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:24.317320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:24.317387 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:24.317698 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:24.317700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:24.317701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:24.317859 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:24.317933 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:24.318355 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 09:40:24.318441 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 09:40:24.319375 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:24.307545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:46243 (local address 127.3.147.3:43003)
0812 09:40:24.307993 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:24.308036 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:24.309354 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:24.309830 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:24.309836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:24.310174 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:24.310616 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:24.310641 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.311424 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.311426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:24.312660 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:24.312670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.313152 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.313156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:24.313377 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:24.313992 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:24.314020 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:24.315461 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:24.316980 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:24.316984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:24.316994 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:24.317233 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:24.317464 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:24.317466 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:24.317467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:24.317611 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:24.317945 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:24.317950 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:24.318162 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 09:40:24.318311 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 09:40:24.318621 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":124,"threads_started":1}
I20260812 09:40:24.320514 7710 heartbeater.cc:347] Connected to a master server at 127.3.147.62:46243
I20260812 09:40:24.320749 7710 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:24.321209 7710 heartbeater.cc:511] Master 127.3.147.62:46243 requested a full tablet report, sending...
I20260812 09:40:24.322191 7178 ts_manager.cc:194] Registered new tserver with Master: df0e66dbe88249b9bcaadea327cf34ff (127.3.147.3:40621)
I20260812 09:40:24.322726 7178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:43003
I20260812 09:40:24.325729 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527624, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527624, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:24.351579 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:24.343706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.3.147.62:46243)
0812 09:40:24.343832 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:24.343836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:24.344038 (+ 202us) server_negotiation.cc:407] Connection header received
0812 09:40:24.344181 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:24.344198 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:24.344251 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:24.344329 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:24.344989 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.345482 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.346288 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:24.346479 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:24.346550 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:24.348936 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:24.348939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:24.348941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:24.348961 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:24.350150 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:24.350544 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:24.350564 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:24.350566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:24.350635 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:24.350885 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:24.350890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:24.350892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:24.351092 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:24.351192 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:24.351380 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 09:40:24.351465 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 09:40:24.351974 7178 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42878
I20260812 09:40:24.352919 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 7251
I20260812 09:40:24.361969 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 7404
I20260812 09:40:24.370098 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 7557
I20260812 09:40:24.377007 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 7143
2026-08-12T09:40:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2233 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7726](info): set up 2 sockets
Aug 12 09:40:24 dist-test-slave-cbbz krb5kdc[7726](info): commencing operation
krb5kdc: starting...
W20260812 09:40:25.542800 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:25 dist-test-slave-cbbz krb5kdc[7726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:25.666848 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:40245
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:40245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:25.778437 7742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:25.778693 7742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:25.778733 7742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:25.782748 7742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:25.782804 7742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:25.782824 7742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:25.782840 7742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:25.782856 7742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:25.788043 7742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:40245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:40245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:25.789063 7742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:25.790023 7742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:25.798429 7748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:25.798434 7752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:25.798435 7749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:25.798724 7742 server_base.cc:1034] running on GCE node
I20260812 09:40:25.799399 7742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:25.800577 7742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:25.801772 7742 hybrid_clock.cc:648] HybridClock initialized: now 1786527625801754 us; error 37 us; skew 500 ppm
Aug 12 09:40:25 dist-test-slave-cbbz krb5kdc[7726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527625, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:25.804497 7742 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:25.805444 7742 webserver.cc:460] Webserver started at http://127.3.147.62:34993/ using document root <none> and password file <none>
I20260812 09:40:25.806021 7742 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:25.806085 7742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:25.806340 7742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:25.808465 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "c1ff8a82285f479695ed7366c8964378"
format_stamp: "Formatted at 2026-08-12 09:40:25 on dist-test-slave-cbbz"
server_key: "fffa4f0d2618b4dcd887ae32aba00b32"
server_key_iv: "8120b83ba3ed352e8db01fa7f72fb396"
server_key_version: "encryptionkey@0"
I20260812 09:40:25.809022 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "c1ff8a82285f479695ed7366c8964378"
format_stamp: "Formatted at 2026-08-12 09:40:25 on dist-test-slave-cbbz"
server_key: "fffa4f0d2618b4dcd887ae32aba00b32"
server_key_iv: "8120b83ba3ed352e8db01fa7f72fb396"
server_key_version: "encryptionkey@0"
I20260812 09:40:25.812639 7742 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:40:25.815007 7762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:25.816270 7742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:25.816416 7742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "c1ff8a82285f479695ed7366c8964378"
format_stamp: "Formatted at 2026-08-12 09:40:25 on dist-test-slave-cbbz"
server_key: "fffa4f0d2618b4dcd887ae32aba00b32"
server_key_iv: "8120b83ba3ed352e8db01fa7f72fb396"
server_key_version: "encryptionkey@0"
I20260812 09:40:25.816526 7742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:25.834672 7742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:25.837527 7742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:25.837731 7742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:25.845238 7742 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:40245
I20260812 09:40:25.845276 7821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:40245 every 8 connection(s)
I20260812 09:40:25.846186 7742 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:25.848933 7822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:25.852677 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 7742
I20260812 09:40:25.852803 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:25.853133 3660 external_mini_cluster.cc:1428] Setting key d5d065270c329ef6f2ad8418818a2118
I20260812 09:40:25.854869 7822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378: Bootstrap starting.
I20260812 09:40:25.857098 7822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:25.857834 7822 log.cc:824] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:25.859637 7822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378: No bootstrap required, opened a new log
Aug 12 09:40:25 dist-test-slave-cbbz krb5kdc[7726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:25.863868 7822 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ff8a82285f479695ed7366c8964378" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40245 } }
I20260812 09:40:25.864076 7822 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:25.864117 7822 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1ff8a82285f479695ed7366c8964378, State: Initialized, Role: FOLLOWER
I20260812 09:40:25.864591 7822 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ff8a82285f479695ed7366c8964378" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40245 } }
I20260812 09:40:25.864751 7822 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:25.864838 7822 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:25.864954 7822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:25.865890 7822 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ff8a82285f479695ed7366c8964378" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40245 } }
I20260812 09:40:25.866230 7822 leader_election.cc:302] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1ff8a82285f479695ed7366c8964378; no voters:
I20260812 09:40:25.866564 7822 leader_election.cc:288] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:25.866626 7829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:25.866865 7829 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [term 1 LEADER]: Becoming Leader. State: Replica: c1ff8a82285f479695ed7366c8964378, State: Running, Role: LEADER
I20260812 09:40:25.867218 7829 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ff8a82285f479695ed7366c8964378" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40245 } }
I20260812 09:40:25.867527 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:25.854542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.3.147.62:40245)
0812 09:40:25.855002 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:25.855025 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:25.855060 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:40:25.855680 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:25.855696 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:25.856004 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:25.856391 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:25.857199 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:25.858277 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:25.859131 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:25.859592 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:25.859771 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:25.862443 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:25.862448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:25.862460 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:25.862485 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:25.864466 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:25.864960 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:25.864966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:25.864972 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:25.865141 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:25.865430 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:25.865435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:25.865437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:25.865816 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:25.866027 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:25.866275 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 09:40:25.866585 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":123,"threads_started":1}
I20260812 09:40:25.867815 7822 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:25.868394 7832 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1ff8a82285f479695ed7366c8964378. Latest consensus state: current_term: 1 leader_uuid: "c1ff8a82285f479695ed7366c8964378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ff8a82285f479695ed7366c8964378" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40245 } } }
I20260812 09:40:25.868516 7832 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:25.869078 7830 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1ff8a82285f479695ed7366c8964378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ff8a82285f479695ed7366c8964378" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40245 } } }
I20260812 09:40:25.869244 7830 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:25.870061 7840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:25.873726 7840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:25.879132 7840 catalog_manager.cc:1347] Generated new cluster ID: 17756869e1574e2a853b357aec5cb950
I20260812 09:40:25.879199 7840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:25.897133 7840 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:25.898049 7840 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:25.905289 7840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1ff8a82285f479695ed7366c8964378: Generated new TSK 0
I20260812 09:40:25.906149 7840 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:25.942945 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40245
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:26.055682 7853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:26.055976 7853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:26.056049 7853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:26.060030 7853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:26.060122 7853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:26.060225 7853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:26.066036 7853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:26.067117 7853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:26.068199 7853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:26.077855 7860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:26.077894 7853 server_base.cc:1034] running on GCE node
W20260812 09:40:26.077862 7861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:26.078045 7863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:26.078526 7853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:26.079218 7853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:26.080400 7853 hybrid_clock.cc:648] HybridClock initialized: now 1786527626080385 us; error 30 us; skew 500 ppm
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:26.083343 7853 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:26.084128 7853 webserver.cc:460] Webserver started at http://127.3.147.1:35193/ using document root <none> and password file <none>
I20260812 09:40:26.084698 7853 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:26.084757 7853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:26.084919 7853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:26.086845 7853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "750c8828b5a142188d37ff7b454fa9d0"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "095cefc0e2c2186d8524b32dcbf70227"
server_key_iv: "42267addef5adeb2c4172ea9e00d9741"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.087365 7853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "750c8828b5a142188d37ff7b454fa9d0"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "095cefc0e2c2186d8524b32dcbf70227"
server_key_iv: "42267addef5adeb2c4172ea9e00d9741"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.091020 7853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 09:40:26.093469 7874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.094676 7853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:26.094771 7853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "750c8828b5a142188d37ff7b454fa9d0"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "095cefc0e2c2186d8524b32dcbf70227"
server_key_iv: "42267addef5adeb2c4172ea9e00d9741"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.094851 7853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:26.104228 7853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:26.107359 7853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:26.107502 7853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:26.107988 7853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:26.108826 7853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:26.108873 7853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.108911 7853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:26.108932 7853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.117986 7853 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:44543
I20260812 09:40:26.118021 8009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:44543 every 8 connection(s)
I20260812 09:40:26.118913 7853 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:26.119783 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 7853
I20260812 09:40:26.119908 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:26.120265 3660 external_mini_cluster.cc:1428] Setting key 2376c5eac8e83247af0e9907e1dd280d
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:26.133997 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:26.120766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:43705 (local address 127.3.147.62:40245)
0812 09:40:26.120917 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:26.120921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:26.122049 (+ 1128us) server_negotiation.cc:407] Connection header received
0812 09:40:26.124245 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:26.124250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:26.124321 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:26.124420 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:26.125504 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.126061 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.127199 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.127437 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.127551 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:26.130362 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:26.130366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:26.130368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:26.130391 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:26.131874 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.132461 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.132469 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.132471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.132554 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.132913 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.132917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.132918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.133096 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:26.133206 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:26.133688 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 09:40:26.133822 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:40:26.134858 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:26.121236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40245 (local address 127.3.147.1:43705)
0812 09:40:26.121882 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:26.121940 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:26.123846 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:26.124570 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:26.124576 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:26.124913 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:26.125363 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:26.125376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.126194 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.126198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:26.127067 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:26.127077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.127593 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.127597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:26.127765 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:26.128498 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:26.128514 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:26.130184 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:26.132002 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:26.132008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:26.132021 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:26.132357 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:26.132651 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:26.132654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:26.132657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:26.132805 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:26.133206 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:26.133214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:26.133451 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 09:40:26.133629 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 09:40:26.133933 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":165,"threads_started":1}
I20260812 09:40:26.136299 8011 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40245
I20260812 09:40:26.136579 8011 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:26.137156 8011 heartbeater.cc:511] Master 127.3.147.62:40245 requested a full tablet report, sending...
I20260812 09:40:26.138783 7779 ts_manager.cc:194] Registered new tserver with Master: 750c8828b5a142188d37ff7b454fa9d0 (127.3.147.1:44543)
I20260812 09:40:26.139921 7779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:43705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:26.151691 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40245
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:26.261963 8016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:26.262252 8016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:26.262331 8016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:26.266393 8016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:26.266476 8016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:26.266558 8016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:26.271867 8016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:26.273226 8016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:26.274333 8016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:40:26.283840 8016 server_base.cc:1034] running on GCE node
W20260812 09:40:26.283866 8024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:26.283725 8023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:26.283725 8027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:26.284490 8016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:26.285137 8016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:26.286350 8016 hybrid_clock.cc:648] HybridClock initialized: now 1786527626286337 us; error 29 us; skew 500 ppm
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:26.289286 8016 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:26.290148 8016 webserver.cc:460] Webserver started at http://127.3.147.2:46111/ using document root <none> and password file <none>
I20260812 09:40:26.290727 8016 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:26.290823 8016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:26.291062 8016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:26.293190 8016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "7a2d22c432544156a60d5edd948c437f"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "5cb801935a57b706360eec89db1fa1e5"
server_key_iv: "be9fdef4bbf4cbdc4fc93a5c871f9e7c"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.293762 8016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "7a2d22c432544156a60d5edd948c437f"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "5cb801935a57b706360eec89db1fa1e5"
server_key_iv: "be9fdef4bbf4cbdc4fc93a5c871f9e7c"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.297485 8016 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:26.300021 8034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.301304 8016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:26.301452 8016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "7a2d22c432544156a60d5edd948c437f"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "5cb801935a57b706360eec89db1fa1e5"
server_key_iv: "be9fdef4bbf4cbdc4fc93a5c871f9e7c"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.301566 8016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:26.311143 8016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:26.314603 8016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:26.314808 8016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:26.315380 8016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:26.316223 8016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:26.316293 8016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.316357 8016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:26.316407 8016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.325389 8016 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46863
I20260812 09:40:26.325423 8171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46863 every 8 connection(s)
I20260812 09:40:26.326251 8016 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:26.327941 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 8016
I20260812 09:40:26.328080 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:26.328428 3660 external_mini_cluster.cc:1428] Setting key 76922bb9707d9d2c1c24c6a3f1358bcf
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:40:26.341750 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:26.328034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:59585 (local address 127.3.147.62:40245)
0812 09:40:26.328219 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:26.328225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:26.329273 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 09:40:26.330894 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:26.330900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:26.330962 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:26.331056 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:26.332424 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.332943 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.334725 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.335036 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.335117 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:26.337995 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:26.337999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:26.338002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:26.338033 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:26.339663 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.340255 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.340258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.340259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.340332 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.340667 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.340672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.340674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.340860 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:26.340975 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:26.341450 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 09:40:26.341576 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:40:26.342571 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:26.328542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40245 (local address 127.3.147.2:59585)
0812 09:40:26.329101 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:26.329164 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:26.330671 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:26.331229 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:26.331237 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:26.331570 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:26.332257 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:26.332279 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.333073 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.333079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:26.334598 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:26.334612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.335210 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.335215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:26.335448 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:26.336079 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:26.336105 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:26.337819 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:26.339798 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:26.339804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:26.339817 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:26.340091 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:26.340415 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:26.340417 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:26.340419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:26.340551 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:26.340974 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:26.340982 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:26.341209 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 09:40:26.341388 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 09:40:26.341692 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":154,"threads_started":1}
I20260812 09:40:26.343719 8172 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40245
I20260812 09:40:26.343966 8172 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:26.344504 8172 heartbeater.cc:511] Master 127.3.147.62:40245 requested a full tablet report, sending...
I20260812 09:40:26.345541 7779 ts_manager.cc:194] Registered new tserver with Master: 7a2d22c432544156a60d5edd948c437f (127.3.147.2:46863)
I20260812 09:40:26.346103 7779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:59585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:26.357576 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40245
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:26.471715 8177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:26.471949 8177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:26.472015 8177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:26.476059 8177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:26.476109 8177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:26.476179 8177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:26.481454 8177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:26.482474 8177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:26.483580 8177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:40:26.493196 8177 server_base.cc:1034] running on GCE node
W20260812 09:40:26.493088 8183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:26.493134 8182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:26.493054 8185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:26.493762 8177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:26.494385 8177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:26.495573 8177 hybrid_clock.cc:648] HybridClock initialized: now 1786527626495559 us; error 27 us; skew 500 ppm
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:26.498377 8177 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:26.499243 8177 webserver.cc:460] Webserver started at http://127.3.147.3:43397/ using document root <none> and password file <none>
I20260812 09:40:26.499763 8177 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:26.499828 8177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:26.499989 8177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:26.502017 8177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "19001bd9a0be4d6aa02df5e84a8839aa"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "abe79b2d286963c3533ed74ecbaf8fa7"
server_key_iv: "f6e1149856871e623c8dd158831938a5"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.502525 8177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "19001bd9a0be4d6aa02df5e84a8839aa"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "abe79b2d286963c3533ed74ecbaf8fa7"
server_key_iv: "f6e1149856871e623c8dd158831938a5"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.506273 8177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:40:26.508703 8192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.509878 8177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:26.509999 8177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "19001bd9a0be4d6aa02df5e84a8839aa"
format_stamp: "Formatted at 2026-08-12 09:40:26 on dist-test-slave-cbbz"
server_key: "abe79b2d286963c3533ed74ecbaf8fa7"
server_key_iv: "f6e1149856871e623c8dd158831938a5"
server_key_version: "encryptionkey@0"
I20260812 09:40:26.510099 8177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:26.533768 8177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:26.537178 8177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:26.537425 8177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:26.538107 8177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:26.539074 8177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:26.539134 8177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.539216 8177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:26.539263 8177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:26.549819 8177 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:46479
I20260812 09:40:26.549844 8322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:46479 every 8 connection(s)
I20260812 09:40:26.550828 8177 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:26.554215 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 8177
I20260812 09:40:26.554339 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:26.554682 3660 external_mini_cluster.cc:1428] Setting key 81cdb107024349e97914fd64e185a58d
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:26.564476 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:26.552843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:50863 (local address 127.3.147.62:40245)
0812 09:40:26.553015 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:26.553019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:26.553889 (+ 870us) server_negotiation.cc:407] Connection header received
0812 09:40:26.555602 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:26.555605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:26.555659 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:26.555769 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:26.556970 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.557508 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.558635 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.558855 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.558969 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:26.561379 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:26.561383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:26.561385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:26.561408 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:26.562621 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.563101 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.563104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.563105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.563177 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.563500 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.563502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.563504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.563663 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:26.563740 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:26.564241 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 09:40:26.564358 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 09:40:26.565307 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:26.553247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40245 (local address 127.3.147.3:50863)
0812 09:40:26.553730 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:26.553781 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:26.555392 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:26.555947 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:26.555954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:26.556362 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:26.556829 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:26.556842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.557638 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.557647 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:26.558502 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:26.558511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.559007 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.559012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:26.559183 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:26.559720 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:26.559736 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:26.561232 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:26.562748 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:26.562753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:26.562763 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:26.563014 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:26.563258 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:26.563260 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:26.563261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:26.563416 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:26.563767 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:26.563772 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:26.563988 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 09:40:26.564220 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 09:40:26.564476 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":136,"threads_started":1}
I20260812 09:40:26.566354 8324 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40245
I20260812 09:40:26.566586 8324 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:26.567070 8324 heartbeater.cc:511] Master 127.3.147.62:40245 requested a full tablet report, sending...
I20260812 09:40:26.568393 7778 ts_manager.cc:194] Registered new tserver with Master: 19001bd9a0be4d6aa02df5e84a8839aa (127.3.147.3:46479)
I20260812 09:40:26.569037 7778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:50863
I20260812 09:40:26.580238 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:26.608795 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:26.598882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.3.147.1:44543)
0812 09:40:26.599137 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:26.599155 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:26.599189 (+ 34us) server_negotiation.cc:407] Connection header received
0812 09:40:26.599310 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:26.599316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:26.599556 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:26.599679 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:26.600630 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.601599 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.602456 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.602814 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.602896 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:26.605502 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:26.605512 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:26.605526 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:26.605576 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:26.607057 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.607473 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.607496 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.607497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.607577 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.607806 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.607808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.607810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.608034 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:26.608282 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:26.608422 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 09:40:26.608579 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[7726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:26.619386 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:26.611891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.3.147.62:40245)
0812 09:40:26.612037 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:26.612040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:26.612055 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:40:26.612226 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:26.612229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:26.612278 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:26.612344 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:26.612868 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.613358 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.614145 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:26.614351 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:26.614429 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:26.616730 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:26.616733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:26.616735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:26.616757 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:26.617920 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.618294 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.618296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.618297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.618365 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:26.618584 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:26.618586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:26.618587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:26.618793 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:26.618895 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:26.619125 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 09:40:26.619217 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 09:40:26.619804 7778 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48964
I20260812 09:40:26.620532 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 7853
I20260812 09:40:26.630002 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 8016
I20260812 09:40:26.637897 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 8177
I20260812 09:40:26.646916 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 7742
2026-08-12T09:40:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2270 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[8339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[8339](info): set up 2 sockets
Aug 12 09:40:26 dist-test-slave-cbbz krb5kdc[8339](info): commencing operation
krb5kdc: starting...
W20260812 09:40:27.757306 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:27 dist-test-slave-cbbz krb5kdc[8339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:27.879449 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44689
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:44689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:27.989197 8353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:27.989490 8353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:27.989564 8353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:27.993595 8353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:27.993681 8353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:27.993718 8353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:27.993781 8353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:27.993814 8353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:27.998996 8353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:44689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:28.000078 8353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:28.001154 8353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:40:28.009756 8353 server_base.cc:1034] running on GCE node
W20260812 09:40:28.009804 8360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:28.009773 8362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:28.009773 8358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:28.010576 8353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:28.011652 8353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:28.012845 8353 hybrid_clock.cc:648] HybridClock initialized: now 1786527628012819 us; error 42 us; skew 500 ppm
Aug 12 09:40:28 dist-test-slave-cbbz krb5kdc[8339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:28.015564 8353 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:28.016390 8353 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:28.016486 8353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:28.016789 8353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:28.018821 8353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "c8a910e0833d4687ac1ed29fe416baa5"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "b1a04354b2f95fd3c7ee4fbbb388444a"
server_key_iv: "d356cdeebff298037c916d0aaf9c1f9e"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.019409 8353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "c8a910e0833d4687ac1ed29fe416baa5"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "b1a04354b2f95fd3c7ee4fbbb388444a"
server_key_iv: "d356cdeebff298037c916d0aaf9c1f9e"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.023037 8353 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 09:40:28.025436 8369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.026579 8353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:28.026719 8353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "c8a910e0833d4687ac1ed29fe416baa5"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "b1a04354b2f95fd3c7ee4fbbb388444a"
server_key_iv: "d356cdeebff298037c916d0aaf9c1f9e"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.026832 8353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:28.042094 8353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:28.045166 8353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:28.045385 8353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:28.052947 8353 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:44689
I20260812 09:40:28.053257 8432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:44689 every 8 connection(s)
I20260812 09:40:28.053689 8353 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:28.055377 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 8353
I20260812 09:40:28.055513 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:28.055825 3660 external_mini_cluster.cc:1428] Setting key 9b8a697e98d375f9edc4659199a26e60
I20260812 09:40:28.056843 8433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:28.062527 8433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5: Bootstrap starting.
Aug 12 09:40:28 dist-test-slave-cbbz krb5kdc[8339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:28.065003 8433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:28.065800 8433 log.cc:824] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:28.068104 8433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5: No bootstrap required, opened a new log
I20260812 09:40:28.070211 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.057507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.3.147.62:44689)
0812 09:40:28.057927 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:28.057951 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:28.057988 (+ 37us) server_negotiation.cc:407] Connection header received
0812 09:40:28.058637 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:28.058661 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:28.058987 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:28.059327 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:28.060189 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.061170 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.062041 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.062518 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.062711 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.065551 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:28.065556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:28.065568 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:28.065595 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:28.067386 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.067888 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.067893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.067897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.068001 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.068335 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.068337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.068338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.068709 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:28.068852 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:28.069178 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 09:40:28.069492 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":153,"threads_started":1}
I20260812 09:40:28.071944 8433 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a910e0833d4687ac1ed29fe416baa5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44689 } }
I20260812 09:40:28.072206 8433 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:28.072270 8433 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a910e0833d4687ac1ed29fe416baa5, State: Initialized, Role: FOLLOWER
I20260812 09:40:28.072767 8433 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a910e0833d4687ac1ed29fe416baa5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44689 } }
I20260812 09:40:28.072893 8433 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:28.072974 8433 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:28.073087 8433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:28.073995 8433 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a910e0833d4687ac1ed29fe416baa5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44689 } }
I20260812 09:40:28.074329 8433 leader_election.cc:302] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a910e0833d4687ac1ed29fe416baa5; no voters:
I20260812 09:40:28.074656 8433 leader_election.cc:288] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:28.074780 8438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:28.075029 8438 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [term 1 LEADER]: Becoming Leader. State: Replica: c8a910e0833d4687ac1ed29fe416baa5, State: Running, Role: LEADER
I20260812 09:40:28.075387 8438 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a910e0833d4687ac1ed29fe416baa5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44689 } }
I20260812 09:40:28.075942 8433 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:28.077214 8439 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8a910e0833d4687ac1ed29fe416baa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a910e0833d4687ac1ed29fe416baa5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44689 } } }
I20260812 09:40:28.077212 8440 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a910e0833d4687ac1ed29fe416baa5. Latest consensus state: current_term: 1 leader_uuid: "c8a910e0833d4687ac1ed29fe416baa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a910e0833d4687ac1ed29fe416baa5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44689 } } }
I20260812 09:40:28.077354 8439 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:28.077358 8440 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:28.077752 8448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:28.080866 8448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:28.086344 8448 catalog_manager.cc:1347] Generated new cluster ID: 7708e908547a4a02a8d75b2dda5ae3d1
I20260812 09:40:28.086416 8448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:28.113782 8448 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:28.114665 8448 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:28.118810 8448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5: Generated new TSK 0
I20260812 09:40:28.119421 8448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:28.155958 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44689
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:28.269591 8462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:28.269883 8462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:28.269970 8462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:28.274013 8462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:28.274103 8462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:28.274196 8462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:28.279471 8462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:28.280522 8462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:28.281517 8462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:28.291132 8470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:28.291121 8467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:28.291185 8462 server_base.cc:1034] running on GCE node
W20260812 09:40:28.291240 8468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:28.291829 8462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:28.292464 8462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:28.293618 8462 hybrid_clock.cc:648] HybridClock initialized: now 1786527628293591 us; error 38 us; skew 500 ppm
Aug 12 09:40:28 dist-test-slave-cbbz krb5kdc[8339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:28.296447 8462 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:28.297329 8462 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:28.297421 8462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:28.297627 8462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:28.299643 8462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "b0b3b5591905438680c03b5b393a4d77"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "270950fcda7afb27051915b5244c6d34"
server_key_iv: "2f4ddb12d5e249d697df8f3182e35c22"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.300280 8462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "b0b3b5591905438680c03b5b393a4d77"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "270950fcda7afb27051915b5244c6d34"
server_key_iv: "2f4ddb12d5e249d697df8f3182e35c22"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.303864 8462 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:28.306186 8478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.307467 8462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:28.307621 8462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "b0b3b5591905438680c03b5b393a4d77"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "270950fcda7afb27051915b5244c6d34"
server_key_iv: "2f4ddb12d5e249d697df8f3182e35c22"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.307739 8462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:28.323230 8462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:28.326435 8462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:28.326630 8462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:28.327184 8462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:28.328053 8462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:28.328130 8462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.328228 8462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:28.328264 8462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.337478 8462 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:37887
I20260812 09:40:28.337496 8609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:37887 every 8 connection(s)
I20260812 09:40:28.338037 8462 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:28.342334 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 8462
I20260812 09:40:28.342460 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:28.342736 3660 external_mini_cluster.cc:1428] Setting key 0d237ad6f050d10d2f333f9f0e66471e
Aug 12 09:40:28 dist-test-slave-cbbz krb5kdc[8339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:28.351509 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.339711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:60239 (local address 127.3.147.62:44689)
0812 09:40:28.339848 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:28.339852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:28.340656 (+ 804us) server_negotiation.cc:407] Connection header received
0812 09:40:28.341997 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:28.342000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:28.342060 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:28.342178 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:28.343521 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.344169 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.345299 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.345537 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.345652 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.348032 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:28.348036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:28.348038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:28.348069 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:28.349532 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.350038 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.350043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.350045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.350126 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.350476 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.350481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.350483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.350653 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:28.350764 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:28.351216 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 09:40:28.351341 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:28.352306 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.340053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44689 (local address 127.3.147.1:60239)
0812 09:40:28.340513 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:28.340553 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:28.341752 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:28.342347 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:28.342354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:28.342801 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:28.343370 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.343387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.344315 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.344319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.345168 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.345177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.345700 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.345704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.345889 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.346453 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:28.346469 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:28.347870 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:28.349655 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:28.349661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:28.349673 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:28.349930 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:28.350229 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:28.350232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:28.350234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:28.350376 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:28.350762 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:28.350768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:28.350989 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 09:40:28.351174 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 09:40:28.351477 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":106,"threads_started":1}
I20260812 09:40:28.353514 8610 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44689
I20260812 09:40:28.353744 8610 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:28.354166 8610 heartbeater.cc:511] Master 127.3.147.62:44689 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:40:28.355815 8391 ts_manager.cc:194] Registered new tserver with Master: b0b3b5591905438680c03b5b393a4d77 (127.3.147.1:37887)
I20260812 09:40:28.357298 8391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:60239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:28.371704 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44689
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:28.483832 8617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:28.484110 8617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:28.484203 8617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:28.488102 8617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:28.488210 8617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:28.488325 8617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:28.493314 8617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:28.494264 8617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:28.495222 8617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:28.504360 8624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:28.504539 8627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:28.504504 8617 server_base.cc:1034] running on GCE node
W20260812 09:40:28.504586 8629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:28.504989 8617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:28.505542 8617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:28.506703 8617 hybrid_clock.cc:648] HybridClock initialized: now 1786527628506687 us; error 27 us; skew 500 ppm
Aug 12 09:40:28 dist-test-slave-cbbz krb5kdc[8339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:28.509505 8617 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:28.510200 8617 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:28.510293 8617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:28.510574 8617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:28.512701 8617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "4ed63250a33c486095f38cfa6352b452"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "15324da21affdb43d5901fafedb6708e"
server_key_iv: "063def20cb7182f0fc8ebda28b2fb05c"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.513252 8617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "4ed63250a33c486095f38cfa6352b452"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "15324da21affdb43d5901fafedb6708e"
server_key_iv: "063def20cb7182f0fc8ebda28b2fb05c"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.516760 8617 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:28.519088 8637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.520272 8617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:28.520393 8617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "4ed63250a33c486095f38cfa6352b452"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "15324da21affdb43d5901fafedb6708e"
server_key_iv: "063def20cb7182f0fc8ebda28b2fb05c"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.520519 8617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:28.541422 8617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:28.544598 8617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:28.544750 8617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:28.545252 8617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:28.546000 8617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:28.546046 8617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.546084 8617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:28.546104 8617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.554945 8617 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:39647
I20260812 09:40:28.554975 8769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:39647 every 8 connection(s)
I20260812 09:40:28.555503 8617 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:28.557515 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 8617
I20260812 09:40:28.557655 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:28.558048 3660 external_mini_cluster.cc:1428] Setting key 3f18678830d5f169ffba3585c79c5aa4
Aug 12 09:40:28 dist-test-slave-cbbz krb5kdc[8339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:40:28.569204 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.557163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:47633 (local address 127.3.147.62:44689)
0812 09:40:28.557313 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:28.557317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:28.558314 (+ 997us) server_negotiation.cc:407] Connection header received
0812 09:40:28.559840 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:28.559843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:28.559897 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:28.559977 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:28.561181 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.561707 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.562842 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.563067 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.563173 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.565636 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:28.565640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:28.565642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:28.565664 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:28.567113 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.567646 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.567650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.567652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.567728 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.568104 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.568109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.568111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.568337 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:28.568447 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:28.568978 (+ 531us) server_negotiation.cc:299] Negotiation successful
0812 09:40:28.569078 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:40:28.570091 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.557585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44689 (local address 127.3.147.2:47633)
0812 09:40:28.558156 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:28.558212 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:28.559630 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:28.560191 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:28.560201 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:28.560569 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:28.561030 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.561044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.561854 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.561858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.562707 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.562717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.563223 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.563227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.563394 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.564034 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:28.564049 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:28.565471 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:28.567245 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:28.567250 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:28.567262 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:28.567540 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:28.567850 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:28.567853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:28.567855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:28.568006 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:28.568447 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:28.568455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:28.568686 (+ 231us) client_negotiation.cc:769] Sending connection context
0812 09:40:28.568927 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 09:40:28.569196 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":156,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:40:28.571270 8770 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44689
I20260812 09:40:28.571522 8770 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:28.572058 8770 heartbeater.cc:511] Master 127.3.147.62:44689 requested a full tablet report, sending...
I20260812 09:40:28.573271 8391 ts_manager.cc:194] Registered new tserver with Master: 4ed63250a33c486095f38cfa6352b452 (127.3.147.2:39647)
I20260812 09:40:28.574143 8391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:47633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:28.587594 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44689
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:28.719365 8775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:28.719627 8775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:28.719715 8775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:28.724211 8775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:28.724265 8775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:28.724370 8775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:28.730180 8775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:28.731351 8775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:28.732462 8775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:40:28.742575 8775 server_base.cc:1034] running on GCE node
W20260812 09:40:28.742532 8780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:28.742516 8784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:28.742522 8781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:28.743235 8775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:28.743825 8775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:28.744987 8775 hybrid_clock.cc:648] HybridClock initialized: now 1786527628744956 us; error 42 us; skew 500 ppm
Aug 12 09:40:28 dist-test-slave-cbbz krb5kdc[8339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:28.748195 8775 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:28.748934 8775 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:28.749033 8775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:28.749244 8775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:28.751331 8775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "f3c141fe111d4f5493c1c412604476d2"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "507d628cea571a7aba7c4559a18dbac9"
server_key_iv: "f9583e244bbf3fe849778347010cd85b"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.751909 8775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "f3c141fe111d4f5493c1c412604476d2"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "507d628cea571a7aba7c4559a18dbac9"
server_key_iv: "f9583e244bbf3fe849778347010cd85b"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.755715 8775 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:40:28.758327 8791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.759649 8775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:28.759789 8775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "f3c141fe111d4f5493c1c412604476d2"
format_stamp: "Formatted at 2026-08-12 09:40:28 on dist-test-slave-cbbz"
server_key: "507d628cea571a7aba7c4559a18dbac9"
server_key_iv: "f9583e244bbf3fe849778347010cd85b"
server_key_version: "encryptionkey@0"
I20260812 09:40:28.759907 8775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:28.775090 8775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:28.778259 8775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:28.778469 8775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:28.779045 8775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:28.779870 8775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:28.779944 8775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.780014 8775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:28.780058 8775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:28.789937 8775 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:35573
I20260812 09:40:28.789963 8917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:35573 every 8 connection(s)
I20260812 09:40:28.790505 8775 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:28.795113 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 8775
I20260812 09:40:28.795241 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:28.795524 3660 external_mini_cluster.cc:1428] Setting key 7a5748a6c07d305090566f738ba790e3
Aug 12 09:40:28 dist-test-slave-cbbz krb5kdc[8339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:28.804761 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.792418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:33237 (local address 127.3.147.62:44689)
0812 09:40:28.792562 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:28.792566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:28.793423 (+ 857us) server_negotiation.cc:407] Connection header received
0812 09:40:28.794816 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:28.794820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:28.794879 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:28.794960 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:28.796403 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.797089 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.798295 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.798509 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.798598 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.801496 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:28.801499 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:28.801502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:28.801526 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:28.802810 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.803288 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.803291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.803292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.803364 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.803652 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.803654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.803655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.803817 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:28.803891 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:28.804467 (+ 576us) server_negotiation.cc:299] Negotiation successful
0812 09:40:28.804585 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 09:40:28.805621 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.792816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44689 (local address 127.3.147.3:33237)
0812 09:40:28.793272 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:28.793313 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:28.794602 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:28.795123 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:28.795130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:28.795621 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:28.796247 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.796265 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.797214 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.797217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.798178 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.798188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.798923 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.798927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.799074 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.799766 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:28.799783 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:28.801337 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:28.802925 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:28.802929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:28.802941 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:28.803195 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:28.803444 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:28.803446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:28.803447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:28.803575 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:28.803904 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:28.803908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:28.804120 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 09:40:28.804421 (+ 301us) client_negotiation.cc:240] Negotiation successful
0812 09:40:28.804749 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":113,"threads_started":1}
I20260812 09:40:28.806677 8918 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44689
I20260812 09:40:28.806963 8918 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:28.807386 8918 heartbeater.cc:511] Master 127.3.147.62:44689 requested a full tablet report, sending...
I20260812 09:40:28.808470 8391 ts_manager.cc:194] Registered new tserver with Master: f3c141fe111d4f5493c1c412604476d2 (127.3.147.3:35573)
I20260812 09:40:28.809234 8391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:33237
I20260812 09:40:28.809495 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:40:28.818425 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.811546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.3.147.62:44689)
0812 09:40:28.811663 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:28.811667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:28.811880 (+ 213us) server_negotiation.cc:407] Connection header received
0812 09:40:28.811971 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:28.811973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:28.812008 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:28.812068 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:28.812774 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.813260 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.814034 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.814230 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.814306 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.815569 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:28.815572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:28.815574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:28.815590 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:28.817075 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.817443 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.817446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.817447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.817514 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:28.817741 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:28.817743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:28.817744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:28.817945 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:28.818042 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:28.818233 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 09:40:28.818319 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 09:40:28.821125 8391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:28.824088 8391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:28.837885 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.833086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.3.147.1:37887)
0812 09:40:28.833524 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:28.833530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:28.833557 (+ 27us) server_negotiation.cc:407] Connection header received
0812 09:40:28.833640 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:28.833646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:28.833862 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:28.834043 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:28.834332 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.835212 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.836656 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.837208 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.837345 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.837498 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:28.837632 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 09:40:28.837722 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":131,"threads_started":1}
I20260812 09:40:28.837877 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.832314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37887 (local address 127.0.0.1:55802)
0812 09:40:28.833244 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:28.833257 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:28.833334 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:28.834030 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:28.834036 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:28.834058 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:28.834197 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.834204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.835349 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.835355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.836531 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.836541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.837344 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.837348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.837457 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.837493 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 09:40:28.837603 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 09:40:28.837721 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"thread_start_us":113,"threads_started":1}
I20260812 09:40:28.838430 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.832244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:39647 (local address 127.0.0.1:60886)
0812 09:40:28.832685 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:28.832729 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:28.832856 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:28.834412 (+ 1556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:28.834415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:28.834435 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:28.834536 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.834542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.835990 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.835993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.836949 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.836959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.838033 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.838037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.838125 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.838151 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 09:40:28.838196 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 09:40:28.838273 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":15,"thread_start_us":88,"threads_started":1}
I20260812 09:40:28.839226 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.832678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.3.147.2:39647)
0812 09:40:28.833953 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:28.833969 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:28.833988 (+ 19us) server_negotiation.cc:407] Connection header received
0812 09:40:28.834054 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:28.834059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:28.834219 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:28.834417 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:28.834641 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.835831 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.837076 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.837897 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.838625 (+ 728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.838768 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:28.838820 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 09:40:28.838904 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1180,"thread_start_us":150,"threads_started":1}
I20260812 09:40:28.839831 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.832943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:35573 (local address 127.0.0.1:60028)
0812 09:40:28.833673 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:28.833686 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:28.833792 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:28.834425 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:28.834428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:28.834446 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:28.834572 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.834578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.835990 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.835993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.838197 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:28.838206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.839404 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.839407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:28.839516 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.839539 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:40:28.839611 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 09:40:28.839683 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":24,"thread_start_us":89,"threads_started":1}
I20260812 09:40:28.840329 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:28.833259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.3.147.3:35573)
0812 09:40:28.833646 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:28.833660 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:28.833789 (+ 129us) server_negotiation.cc:407] Connection header received
0812 09:40:28.833904 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:28.833910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:28.834084 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:28.834246 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:28.834711 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.835831 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.838324 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:28.839097 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:28.839210 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:28.839344 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:28.840027 (+ 683us) server_negotiation.cc:299] Negotiation successful
0812 09:40:28.840128 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":162,"threads_started":1}
I20260812 09:40:28.841557 8533 tablet_service.cc:1467] Processing CreateTablet for tablet f1e794a0d41c4178b817505c55eb9a0e (DEFAULT_TABLE table=test-table [id=68ecbbdcf1be446490a4725ea32c8be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:28.842304 8693 tablet_service.cc:1467] Processing CreateTablet for tablet f1e794a0d41c4178b817505c55eb9a0e (DEFAULT_TABLE table=test-table [id=68ecbbdcf1be446490a4725ea32c8be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:28.842757 8533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1e794a0d41c4178b817505c55eb9a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:28.843293 8693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1e794a0d41c4178b817505c55eb9a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:28.844491 8847 tablet_service.cc:1467] Processing CreateTablet for tablet f1e794a0d41c4178b817505c55eb9a0e (DEFAULT_TABLE table=test-table [id=68ecbbdcf1be446490a4725ea32c8be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:28.845213 8847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1e794a0d41c4178b817505c55eb9a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:28.850337 8936 tablet_bootstrap.cc:492] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: Bootstrap starting.
I20260812 09:40:28.851161 8937 tablet_bootstrap.cc:492] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: Bootstrap starting.
I20260812 09:40:28.853314 8937 tablet_bootstrap.cc:654] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:28.853951 8936 tablet_bootstrap.cc:654] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:28.854866 8937 log.cc:824] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:28.855321 8936 log.cc:824] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:28.855975 8939 tablet_bootstrap.cc:492] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77: Bootstrap starting.
I20260812 09:40:28.857174 8937 tablet_bootstrap.cc:492] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: No bootstrap required, opened a new log
I20260812 09:40:28.857395 8937 ts_tablet_manager.cc:1397] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:40:28.857412 8936 tablet_bootstrap.cc:492] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: No bootstrap required, opened a new log
I20260812 09:40:28.857591 8936 ts_tablet_manager.cc:1397] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 09:40:28.858135 8939 tablet_bootstrap.cc:654] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:28.858940 8939 log.cc:824] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:28.860949 8939 tablet_bootstrap.cc:492] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77: No bootstrap required, opened a new log
I20260812 09:40:28.860811 8937 raft_consensus.cc:348] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:28.861150 8939 ts_tablet_manager.cc:1397] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:40:28.861147 8937 raft_consensus.cc:374] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:28.861208 8937 raft_consensus.cc:729] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c141fe111d4f5493c1c412604476d2, State: Initialized, Role: FOLLOWER
I20260812 09:40:28.861588 8936 raft_consensus.cc:348] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:28.861690 8937 consensus_queue.cc:260] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:28.861830 8936 raft_consensus.cc:374] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:28.861904 8936 raft_consensus.cc:729] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed63250a33c486095f38cfa6352b452, State: Initialized, Role: FOLLOWER
I20260812 09:40:28.862367 8918 heartbeater.cc:503] Master 127.3.147.62:44689 was elected leader, sending a full tablet report...
I20260812 09:40:28.862774 8937 ts_tablet_manager.cc:1428] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 09:40:28.863430 8936 consensus_queue.cc:260] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:28.864076 8770 heartbeater.cc:503] Master 127.3.147.62:44689 was elected leader, sending a full tablet report...
I20260812 09:40:28.865335 8936 ts_tablet_manager.cc:1428] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 09:40:28.865296 8939 raft_consensus.cc:348] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:28.865549 8939 raft_consensus.cc:374] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:28.865622 8939 raft_consensus.cc:729] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b3b5591905438680c03b5b393a4d77, State: Initialized, Role: FOLLOWER
I20260812 09:40:28.866058 8939 consensus_queue.cc:260] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:28.866864 8939 ts_tablet_manager.cc:1428] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:40:28.867375 8610 heartbeater.cc:503] Master 127.3.147.62:44689 was elected leader, sending a full tablet report...
I20260812 09:40:29.004527 8945 raft_consensus.cc:482] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:29.004822 8945 raft_consensus.cc:504] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:29.006071 8945 leader_election.cc:288] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed63250a33c486095f38cfa6352b452 (127.3.147.2:39647), b0b3b5591905438680c03b5b393a4d77 (127.3.147.1:37887)
I20260812 09:40:29.010152 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.006688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:36347 (local address 127.3.147.1:37887)
0812 09:40:29.006818 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:29.006821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:29.006861 (+ 40us) server_negotiation.cc:407] Connection header received
0812 09:40:29.006914 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:29.006916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:29.006958 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:29.007060 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:29.007393 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.007975 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.009077 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.009536 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.009662 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:29.009736 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:29.009943 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 09:40:29.010023 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:40:29.010149 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.006428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37887 (local address 127.3.147.3:36347)
0812 09:40:29.006657 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.006674 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:29.006778 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:29.007109 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:29.007112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:29.007133 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:29.007259 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:29.007265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.008100 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.008105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:29.008940 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:29.008949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.009692 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.009697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:29.009816 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:29.009834 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 09:40:29.009926 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 09:40:29.009999 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260812 09:40:29.010315 8948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.006428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:39647 (local address 127.3.147.3:36003)
0812 09:40:29.006914 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.006925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:29.007042 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:29.007441 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:29.007443 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:29.007457 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:29.007551 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:29.007554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.008364 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.008368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:29.009171 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:29.009179 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.009858 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.009862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:29.009973 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:29.009991 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 09:40:29.010089 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 09:40:29.010164 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":40,"thread_start_us":142,"threads_started":1}
I20260812 09:40:29.010357 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.006562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:36003 (local address 127.3.147.2:39647)
0812 09:40:29.006703 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:29.006710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:29.007024 (+ 314us) server_negotiation.cc:407] Connection header received
0812 09:40:29.007193 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:29.007196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:29.007256 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:29.007317 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:29.007674 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.008236 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.009314 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.009723 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.009854 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:29.009918 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:29.010136 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 09:40:29.010235 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 09:40:29.011021 8554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1e794a0d41c4178b817505c55eb9a0e" candidate_uuid: "f3c141fe111d4f5493c1c412604476d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b3b5591905438680c03b5b393a4d77" is_pre_election: true
I20260812 09:40:29.011029 8718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1e794a0d41c4178b817505c55eb9a0e" candidate_uuid: "f3c141fe111d4f5493c1c412604476d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed63250a33c486095f38cfa6352b452" is_pre_election: true
I20260812 09:40:29.011363 8718 raft_consensus.cc:2454] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c141fe111d4f5493c1c412604476d2 in term 0.
I20260812 09:40:29.011385 8554 raft_consensus.cc:2454] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c141fe111d4f5493c1c412604476d2 in term 0.
I20260812 09:40:29.011924 8792 leader_election.cc:302] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ed63250a33c486095f38cfa6352b452, b0b3b5591905438680c03b5b393a4d77, f3c141fe111d4f5493c1c412604476d2; no voters:
I20260812 09:40:29.012373 8945 raft_consensus.cc:2781] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:29.012454 8945 raft_consensus.cc:482] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:29.012518 8945 raft_consensus.cc:3037] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 0 FOLLOWER]: Advancing to term 1
W20260812 09:40:29.041404 8919 tablet.cc:2369] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:40:29.056593 8771 tablet.cc:2369] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:40:29.061482 8945 raft_consensus.cc:504] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:29.062040 8945 leader_election.cc:288] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed63250a33c486095f38cfa6352b452 (127.3.147.2:39647), b0b3b5591905438680c03b5b393a4d77 (127.3.147.1:37887)
I20260812 09:40:29.062542 8718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1e794a0d41c4178b817505c55eb9a0e" candidate_uuid: "f3c141fe111d4f5493c1c412604476d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed63250a33c486095f38cfa6352b452"
I20260812 09:40:29.062700 8718 raft_consensus.cc:3037] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:29.062706 8554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1e794a0d41c4178b817505c55eb9a0e" candidate_uuid: "f3c141fe111d4f5493c1c412604476d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b3b5591905438680c03b5b393a4d77"
I20260812 09:40:29.062882 8554 raft_consensus.cc:3037] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:29.064245 8718 raft_consensus.cc:2454] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c141fe111d4f5493c1c412604476d2 in term 1.
I20260812 09:40:29.064735 8554 raft_consensus.cc:2454] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c141fe111d4f5493c1c412604476d2 in term 1.
I20260812 09:40:29.064776 8792 leader_election.cc:302] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed63250a33c486095f38cfa6352b452, f3c141fe111d4f5493c1c412604476d2; no voters:
I20260812 09:40:29.064952 8945 raft_consensus.cc:2781] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:29.065162 8945 raft_consensus.cc:686] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 1 LEADER]: Becoming Leader. State: Replica: f3c141fe111d4f5493c1c412604476d2, State: Running, Role: LEADER
I20260812 09:40:29.065524 8945 consensus_queue.cc:237] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } }
I20260812 09:40:29.069200 8390 catalog_manager.cc:5495] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c141fe111d4f5493c1c412604476d2 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "f3c141fe111d4f5493c1c412604476d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c141fe111d4f5493c1c412604476d2" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 35573 } health_report { overall_health: HEALTHY } } }
W20260812 09:40:29.089358 8612 tablet.cc:2369] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:40:29.132154 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.128507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.3.147.3:35573)
0812 09:40:29.128644 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:29.128648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:29.128723 (+ 75us) server_negotiation.cc:407] Connection header received
0812 09:40:29.128851 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:29.128853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:29.128912 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:29.129001 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:29.129234 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.129736 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.130794 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:29.130996 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:29.131075 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:29.131398 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:29.131791 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:29.131880 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:29.131954 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 09:40:29.132003 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:40:29.138028 8718 raft_consensus.cc:1261] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 1 FOLLOWER]: Refusing update from remote peer f3c141fe111d4f5493c1c412604476d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:29.138198 8554 raft_consensus.cc:1261] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 1 FOLLOWER]: Refusing update from remote peer f3c141fe111d4f5493c1c412604476d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:29.138785 8950 consensus_queue.cc:1035] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed63250a33c486095f38cfa6352b452" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 39647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:29.138943 8945 consensus_queue.cc:1035] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b3b5591905438680c03b5b393a4d77" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:29.158001 8386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:29.158164 8386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:29.160594 8386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8a910e0833d4687ac1ed29fe416baa5: Sending DeleteTablet for 3 replicas of tablet f1e794a0d41c4178b817505c55eb9a0e
I20260812 09:40:29.161518 8693 tablet_service.cc:1514] Processing DeleteTablet for tablet f1e794a0d41c4178b817505c55eb9a0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:29 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:60886
I20260812 09:40:29.161589 8533 tablet_service.cc:1514] Processing DeleteTablet for tablet f1e794a0d41c4178b817505c55eb9a0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:29 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:55802
I20260812 09:40:29.162139 8964 tablet_replica.cc:330] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77: stopping tablet replica
I20260812 09:40:29.162158 8847 tablet_service.cc:1514] Processing DeleteTablet for tablet f1e794a0d41c4178b817505c55eb9a0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:29 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:60028
I20260812 09:40:29.162506 8964 raft_consensus.cc:2229] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:29.162595 8967 tablet_replica.cc:330] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: stopping tablet replica
I20260812 09:40:29.162911 8964 raft_consensus.cc:2258] T f1e794a0d41c4178b817505c55eb9a0e P b0b3b5591905438680c03b5b393a4d77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:29.163038 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 8462
I20260812 09:40:29.163062 8967 raft_consensus.cc:2229] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:29.163496 8967 raft_consensus.cc:2258] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:29.166468 8967 ts_tablet_manager.cc:1905] T f1e794a0d41c4178b817505c55eb9a0e P f3c141fe111d4f5493c1c412604476d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:29.167153 8965 tablet_replica.cc:330] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: stopping tablet replica
I20260812 09:40:29.167605 8965 raft_consensus.cc:2229] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:29.168001 8965 raft_consensus.cc:2258] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:29.169678 8965 ts_tablet_manager.cc:1905] T f1e794a0d41c4178b817505c55eb9a0e P 4ed63250a33c486095f38cfa6352b452: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 09:40:29.174465 8374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:37887 (error 108)
I20260812 09:40:29.175552 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 8617
I20260812 09:40:29.175851 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.175405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37887 (local address 127.0.0.1:55812)
0812 09:40:29.175654 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.175739 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260812 09:40:29.176045 8374 catalog_manager.cc:4584] TS b0b3b5591905438680c03b5b393a4d77 (127.3.147.1:37887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1e794a0d41c4178b817505c55eb9a0e: Network error: Client connection negotiation failed: client connection to 127.3.147.1:37887: connect: Connection refused (error 111)
I20260812 09:40:29.229530 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.228966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37887 (local address 127.0.0.1:55824)
0812 09:40:29.229158 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.229416 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 09:40:29.230521 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.230215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37887 (local address 127.0.0.1:55834)
0812 09:40:29.230379 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.230422 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260812 09:40:29.241187 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.240797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:39647 (local address 127.0.0.1:60900)
0812 09:40:29.240942 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.241054 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.2:39647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 09:40:29.241297 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 8775
I20260812 09:40:29.277351 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.276894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:39647 (local address 127.0.0.1:60904)
0812 09:40:29.277099 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.277187 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.2:39647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 09:40:29.278396 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.278089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:39647 (local address 127.0.0.1:60910)
0812 09:40:29.278238 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.278306 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.2:39647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 09:40:29.280414 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 8353
I20260812 09:40:29.280640 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.280393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:35573 (local address 127.0.0.1:60046)
0812 09:40:29.280542 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.280585 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.3:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
2026-08-12T09:40:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2634 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T09:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:29Z Disabled control of system clock
I20260812 09:40:29.315155 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41461
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:41461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 09:40:29.427703 8973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:29.427964 8973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:29.428001 8973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:29.432207 8973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:29.432266 8973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:29.432282 8973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:29.432300 8973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:29.432315 8973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:29.437076 8973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:41461
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:29.438108 8973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:29.439498 8973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:29.448765 8978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:29.448776 8982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:29.448776 8979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:29.449159 8973 server_base.cc:1034] running on GCE node
I20260812 09:40:29.449908 8973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:29.451094 8973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:29.452270 8973 hybrid_clock.cc:648] HybridClock initialized: now 1786527629452250 us; error 30 us; skew 500 ppm
I20260812 09:40:29.454278 8973 webserver.cc:460] Webserver started at http://127.3.147.62:44651/ using document root <none> and password file <none>
I20260812 09:40:29.454879 8973 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:29.454977 8973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:29.455217 8973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:29.457424 8973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "d79ea4e8b91345afadd65d3c0e780808"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "9e8e765247b44c7a7b8e801b17e9186c"
server_key_iv: "272a88724dd90616e05f52ec5fe0b4cf"
server_key_version: "encryptionkey@0"
I20260812 09:40:29.458189 8973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "d79ea4e8b91345afadd65d3c0e780808"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "9e8e765247b44c7a7b8e801b17e9186c"
server_key_iv: "272a88724dd90616e05f52ec5fe0b4cf"
server_key_version: "encryptionkey@0"
I20260812 09:40:29.462021 8973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:40:29.464855 8990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:29.466120 8973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:29.466414 8973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "d79ea4e8b91345afadd65d3c0e780808"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "9e8e765247b44c7a7b8e801b17e9186c"
server_key_iv: "272a88724dd90616e05f52ec5fe0b4cf"
server_key_version: "encryptionkey@0"
I20260812 09:40:29.466540 8973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:29.471474 8973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:29.472096 8973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:29.472345 8973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:29.479846 8973 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:41461
I20260812 09:40:29.479861 9055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:41461 every 8 connection(s)
I20260812 09:40:29.481238 8973 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:29.484709 9056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:29.490129 9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808: Bootstrap starting.
I20260812 09:40:29.491168 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 8973
I20260812 09:40:29.491261 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:29.491585 3660 external_mini_cluster.cc:1428] Setting key b4a45c786d9e665051a4aa313dc33246
I20260812 09:40:29.492925 9056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:29.493728 9056 log.cc:824] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:29.495836 9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808: No bootstrap required, opened a new log
I20260812 09:40:29.497252 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.492856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.3.147.62:41461)
0812 09:40:29.493505 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:29.493520 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:29.493554 (+ 34us) server_negotiation.cc:407] Connection header received
0812 09:40:29.494275 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:29.494294 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:29.494580 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:29.494931 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:29.495760 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:29.495764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:29.495776 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:29.495786 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:29.495841 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:29.495857 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:29.495992 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:29.496167 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 09:40:29.496421 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":130,"threads_started":1}
I20260812 09:40:29.499962 9056 raft_consensus.cc:348] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79ea4e8b91345afadd65d3c0e780808" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41461 } }
I20260812 09:40:29.500223 9056 raft_consensus.cc:374] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:29.500304 9056 raft_consensus.cc:729] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79ea4e8b91345afadd65d3c0e780808, State: Initialized, Role: FOLLOWER
I20260812 09:40:29.500720 9056 consensus_queue.cc:260] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79ea4e8b91345afadd65d3c0e780808" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41461 } }
I20260812 09:40:29.500905 9056 raft_consensus.cc:388] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:29.500994 9056 raft_consensus.cc:482] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:29.501097 9056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:29.583974 9056 raft_consensus.cc:504] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79ea4e8b91345afadd65d3c0e780808" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41461 } }
I20260812 09:40:29.584560 9056 leader_election.cc:302] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d79ea4e8b91345afadd65d3c0e780808; no voters:
I20260812 09:40:29.584969 9056 leader_election.cc:288] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:29.585180 9061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:29.585544 9061 raft_consensus.cc:686] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [term 1 LEADER]: Becoming Leader. State: Replica: d79ea4e8b91345afadd65d3c0e780808, State: Running, Role: LEADER
I20260812 09:40:29.586041 9061 consensus_queue.cc:237] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79ea4e8b91345afadd65d3c0e780808" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41461 } }
I20260812 09:40:29.586464 9056 sys_catalog.cc:564] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:29.588315 9062 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d79ea4e8b91345afadd65d3c0e780808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79ea4e8b91345afadd65d3c0e780808" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41461 } } }
I20260812 09:40:29.588456 9062 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:29.588789 9063 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d79ea4e8b91345afadd65d3c0e780808. Latest consensus state: current_term: 1 leader_uuid: "d79ea4e8b91345afadd65d3c0e780808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79ea4e8b91345afadd65d3c0e780808" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41461 } } }
I20260812 09:40:29.589748 9063 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808 [sys.catalog]: This master's current role is: LEADER
W20260812 09:40:29.592327 9079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:40:29.592417 9079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:40:29.592521 9078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:29.593257 9078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:29.598745 9078 catalog_manager.cc:1347] Generated new cluster ID: cf1ddf2ec6a64cefb3c52564016598d6
I20260812 09:40:29.598814 9078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:29.622349 9078 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:29.622838 9078 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:29.631706 9078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808: Generated new TSK 0
I20260812 09:40:29.632470 9078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:40:29.639670 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41461
--builtin_ntp_servers=127.3.147.20:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 09:40:29.770408 9083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:29.770691 9083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:29.770763 9083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:29.775074 9083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:29.775139 9083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:29.775199 9083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:29.780488 9083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:29.781622 9083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:29.783123 9083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:29.795284 9091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:29.795293 9089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:29.795599 9094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:29.795921 9083 server_base.cc:1034] running on GCE node
I20260812 09:40:29.796353 9083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:29.796926 9083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:29.798085 9083 hybrid_clock.cc:648] HybridClock initialized: now 1786527629798058 us; error 58 us; skew 500 ppm
I20260812 09:40:29.799830 9083 webserver.cc:460] Webserver started at http://127.3.147.1:34261/ using document root <none> and password file <none>
I20260812 09:40:29.800455 9083 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:29.800523 9083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:29.800693 9083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:29.802676 9083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "1a39ff32af1b416d9a05b33bb6876eea"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "3b7970fbe172ecac9ab240f47151a673"
server_key_iv: "77fbef8dcec58429a22932dd2fe63503"
server_key_version: "encryptionkey@0"
I20260812 09:40:29.803185 9083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "1a39ff32af1b416d9a05b33bb6876eea"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "3b7970fbe172ecac9ab240f47151a673"
server_key_iv: "77fbef8dcec58429a22932dd2fe63503"
server_key_version: "encryptionkey@0"
I20260812 09:40:29.807194 9083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 09:40:29.809669 9102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:29.810932 9083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 09:40:29.811030 9083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "1a39ff32af1b416d9a05b33bb6876eea"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "3b7970fbe172ecac9ab240f47151a673"
server_key_iv: "77fbef8dcec58429a22932dd2fe63503"
server_key_version: "encryptionkey@0"
I20260812 09:40:29.811110 9083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:29.815529 9083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:29.816077 9083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:29.816298 9083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:29.816797 9083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:29.817718 9083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:29.817763 9083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:29.817808 9083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:29.817823 9083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:29.827268 9083 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:39267
I20260812 09:40:29.827446 9232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:39267 every 8 connection(s)
I20260812 09:40:29.828263 9083 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:29.834833 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.830190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:59815 (local address 127.3.147.62:41461)
0812 09:40:29.830358 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:29.830362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:29.831428 (+ 1066us) server_negotiation.cc:407] Connection header received
0812 09:40:29.832965 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:29.832968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:29.833006 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:29.833084 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:29.834453 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:29.834456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:29.834458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:29.834466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:29.834481 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:29.834488 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:29.834560 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:29.834692 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 09:40:29.834726 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 09:40:29.835685 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:29.830561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41461 (local address 127.3.147.1:59815)
0812 09:40:29.831275 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:29.831316 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:29.832758 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:29.833247 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:29.833254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:29.833338 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:29.834213 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:40:29.834215 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:29.834255 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:40:29.834321 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:29.834561 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:29.834569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:29.834572 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 09:40:29.834710 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 09:40:29.834926 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":128,"threads_started":1}
I20260812 09:40:29.837039 9234 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41461
I20260812 09:40:29.837291 9234 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:29.837771 9234 heartbeater.cc:511] Master 127.3.147.62:41461 requested a full tablet report, sending...
I20260812 09:40:29.837903 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 9083
I20260812 09:40:29.838032 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:29.838385 3660 external_mini_cluster.cc:1428] Setting key 11535ad1cb58c686b0986ade5b7b8c59
I20260812 09:40:29.839352 9011 ts_manager.cc:194] Registered new tserver with Master: 1a39ff32af1b416d9a05b33bb6876eea (127.3.147.1:39267)
I20260812 09:40:29.840583 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41461
--builtin_ntp_servers=127.3.147.20:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 09:40:29.962004 9237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:29.962278 9237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:29.962319 9237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:29.966310 9237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:29.966363 9237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:29.966418 9237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:29.971619 9237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:29.972673 9237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:29.974045 9237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:29.984100 9245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:29.984153 9242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:29.984217 9243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:29.984328 9237 server_base.cc:1034] running on GCE node
I20260812 09:40:29.985046 9237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:29.985630 9237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:29.986847 9237 hybrid_clock.cc:648] HybridClock initialized: now 1786527629986822 us; error 31 us; skew 500 ppm
I20260812 09:40:29.988821 9237 webserver.cc:460] Webserver started at http://127.3.147.2:39667/ using document root <none> and password file <none>
I20260812 09:40:29.989445 9237 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:29.989544 9237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:29.989780 9237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:29.991827 9237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "da810f3a10284ac885513631055427b6"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "eb518c99394790d2b7156d67c4fe5e45"
server_key_iv: "a433ee36d8cbc9943562bcd800302c8e"
server_key_version: "encryptionkey@0"
I20260812 09:40:29.992455 9237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "da810f3a10284ac885513631055427b6"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "eb518c99394790d2b7156d67c4fe5e45"
server_key_iv: "a433ee36d8cbc9943562bcd800302c8e"
server_key_version: "encryptionkey@0"
I20260812 09:40:29.996619 9237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 09:40:30.000209 9253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:30.001756 9237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:30.001902 9237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "da810f3a10284ac885513631055427b6"
format_stamp: "Formatted at 2026-08-12 09:40:29 on dist-test-slave-cbbz"
server_key: "eb518c99394790d2b7156d67c4fe5e45"
server_key_iv: "a433ee36d8cbc9943562bcd800302c8e"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.002015 9237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:30.006675 9237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:30.007287 9237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:30.007467 9237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:30.008023 9237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:30.008962 9237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:30.009011 9237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:30.009074 9237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:30.009109 9237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:30.018083 9237 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:43741
I20260812 09:40:30.018141 9389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:43741 every 8 connection(s)
I20260812 09:40:30.019016 9237 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:30.025225 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.020980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:46889 (local address 127.3.147.62:41461)
0812 09:40:30.021121 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.021124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.022243 (+ 1119us) server_negotiation.cc:407] Connection header received
0812 09:40:30.023549 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.023551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.023588 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.023664 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.024841 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.024844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.024846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.024857 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.024876 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.024885 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.024963 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.025080 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.025117 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:40:30.026352 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.021382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41461 (local address 127.3.147.2:46889)
0812 09:40:30.022093 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.022134 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:30.023364 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:30.023837 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:30.023844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:30.023929 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:30.024566 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:40:30.024567 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:30.024610 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:40:30.024699 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:30.024960 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:30.024967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:30.024971 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 09:40:30.025125 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 09:40:30.025319 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":110,"threads_started":1}
I20260812 09:40:30.027513 9390 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41461
I20260812 09:40:30.027817 9390 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:30.028038 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 9237
I20260812 09:40:30.028194 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:30.028378 9390 heartbeater.cc:511] Master 127.3.147.62:41461 requested a full tablet report, sending...
I20260812 09:40:30.028535 3660 external_mini_cluster.cc:1428] Setting key c17ba6b3136dbaf89d3f474deed4746f
I20260812 09:40:30.029539 9010 ts_manager.cc:194] Registered new tserver with Master: da810f3a10284ac885513631055427b6 (127.3.147.2:43741)
I20260812 09:40:30.030601 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41461
--builtin_ntp_servers=127.3.147.20:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 09:40:30.144445 9393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:30.144701 9393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:30.144740 9393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:30.148831 9393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:30.148885 9393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:30.148970 9393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:30.153946 9393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:30.155054 9393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:30.156541 9393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:30.166019 9400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:30.166042 9403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:30.166224 9393 server_base.cc:1034] running on GCE node
W20260812 09:40:30.166033 9399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:30.166806 9393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:30.167457 9393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:30.168672 9393 hybrid_clock.cc:648] HybridClock initialized: now 1786527630168656 us; error 32 us; skew 500 ppm
I20260812 09:40:30.170661 9393 webserver.cc:460] Webserver started at http://127.3.147.3:42387/ using document root <none> and password file <none>
I20260812 09:40:30.171289 9393 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:30.171365 9393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:30.171663 9393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:30.173893 9393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "84e04894e3bd484d828972d8f14dfd59"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "7bd21c7c064117cdfa1cdfe519589342"
server_key_iv: "1576bb025aa955bdcc81f29da7f8d6ff"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.174686 9393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "84e04894e3bd484d828972d8f14dfd59"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "7bd21c7c064117cdfa1cdfe519589342"
server_key_iv: "1576bb025aa955bdcc81f29da7f8d6ff"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.178709 9393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 09:40:30.181337 9409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:30.182566 9393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:30.182699 9393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "84e04894e3bd484d828972d8f14dfd59"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "7bd21c7c064117cdfa1cdfe519589342"
server_key_iv: "1576bb025aa955bdcc81f29da7f8d6ff"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.182803 9393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:30.187475 9393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:30.188117 9393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:30.188334 9393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:30.188930 9393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:30.189874 9393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:30.189922 9393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:30.189960 9393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:30.189981 9393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:30.198547 9393 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42253
I20260812 09:40:30.198572 9538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42253 every 8 connection(s)
I20260812 09:40:30.199405 9393 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:30.205751 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.201549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:58385 (local address 127.3.147.62:41461)
0812 09:40:30.201714 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.201719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.202620 (+ 901us) server_negotiation.cc:407] Connection header received
0812 09:40:30.203949 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.203952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.203997 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.204107 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.205254 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.205257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.205260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.205282 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.205307 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.205316 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.205420 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.205573 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.205619 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 09:40:30.206662 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.201823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41461 (local address 127.3.147.3:58385)
0812 09:40:30.202468 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.202507 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:30.203726 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:30.204287 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:30.204294 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:30.204378 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:30.204998 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:40:30.204999 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:30.205037 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:40:30.205101 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:30.205418 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:30.205429 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:30.205438 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 09:40:30.205636 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 09:40:30.205875 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":141,"threads_started":1}
I20260812 09:40:30.207142 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 9393
I20260812 09:40:30.207284 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:30.207573 3660 external_mini_cluster.cc:1428] Setting key 51f836562c6b3de7d036f5cf3372b968
I20260812 09:40:30.207824 9539 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41461
I20260812 09:40:30.208057 9539 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:30.208509 9539 heartbeater.cc:511] Master 127.3.147.62:41461 requested a full tablet report, sending...
I20260812 09:40:30.209528 9011 ts_manager.cc:194] Registered new tserver with Master: 84e04894e3bd484d828972d8f14dfd59 (127.3.147.3:42253)
I20260812 09:40:30.221400 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:40:30.224998 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.223384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.3.147.62:41461)
0812 09:40:30.223565 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.223570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.223720 (+ 150us) server_negotiation.cc:407] Connection header received
0812 09:40:30.223789 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.223792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.223826 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.223901 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.224617 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.224620 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.224622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.224633 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.224654 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.224662 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.224737 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.224819 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.224852 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 09:40:30.227901 9011 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:30.231038 9011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:30.243427 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.239988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:43741 (local address 127.0.0.1:39806)
0812 09:40:30.240965 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.240981 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:30.241069 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:30.241860 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:30.241869 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:30.241953 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:30.242543 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:40:30.242545 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:30.242585 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:40:30.242655 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:30.243066 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:30.243076 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:30.243081 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 09:40:30.243195 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 09:40:30.243265 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":171,"threads_started":1}
I20260812 09:40:30.243827 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.239878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42253 (local address 127.0.0.1:58298)
0812 09:40:30.240739 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.240791 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:30.240915 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:30.242236 (+ 1321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:30.242242 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:30.242259 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:30.243184 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:40:30.243186 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:30.243196 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:40:30.243221 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:30.243592 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:30.243595 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:30.243599 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 09:40:30.243668 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 09:40:30.243697 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":71,"threads_started":1}
I20260812 09:40:30.243880 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.240597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.3.147.2:43741)
0812 09:40:30.240943 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.240954 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.241165 (+ 211us) server_negotiation.cc:407] Connection header received
0812 09:40:30.241223 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.241228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.241433 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.241570 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.242810 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.242819 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.242832 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.242845 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.242888 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.242922 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.243025 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.243618 (+ 593us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.243701 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260812 09:40:30.244266 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.240211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58298 (local address 127.3.147.3:42253)
0812 09:40:30.240735 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.240748 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.241042 (+ 294us) server_negotiation.cc:407] Connection header received
0812 09:40:30.241116 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.241123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.241319 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.241505 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.243348 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.243354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.243365 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.243376 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.243414 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.243445 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.243547 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.244009 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.244085 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":124,"threads_started":1}
I20260812 09:40:30.244976 9558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.241213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.3.147.1:39267)
0812 09:40:30.241610 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.241623 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.241692 (+ 69us) server_negotiation.cc:407] Connection header received
0812 09:40:30.241832 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.241847 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.242007 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.242155 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.244225 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.244232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.244250 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.244264 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.244303 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.244362 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.244486 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.244697 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.244778 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":152,"threads_started":1}
I20260812 09:40:30.245406 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.240971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:39267 (local address 127.0.0.1:58968)
0812 09:40:30.241613 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.241627 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:30.241705 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:30.242834 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:30.242837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:30.242853 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:30.243475 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:40:30.243477 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:30.243507 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:40:30.243767 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:30.244536 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:30.244539 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:30.244543 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 09:40:30.245249 (+ 706us) client_negotiation.cc:240] Negotiation successful
0812 09:40:30.245294 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":124,"threads_started":1}
I20260812 09:40:30.247188 9463 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7ddd2ce9f044319d7b9a665409a452 (DEFAULT_TABLE table=test-table [id=31277f0cd55a408bab844472cef90d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:30.247272 9312 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7ddd2ce9f044319d7b9a665409a452 (DEFAULT_TABLE table=test-table [id=31277f0cd55a408bab844472cef90d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:30.248499 9463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7ddd2ce9f044319d7b9a665409a452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:30.248747 9312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7ddd2ce9f044319d7b9a665409a452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:30.249330 9158 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7ddd2ce9f044319d7b9a665409a452 (DEFAULT_TABLE table=test-table [id=31277f0cd55a408bab844472cef90d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:30.250386 9158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7ddd2ce9f044319d7b9a665409a452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:30.254987 9559 tablet_bootstrap.cc:492] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: Bootstrap starting.
I20260812 09:40:30.256915 9560 tablet_bootstrap.cc:492] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6: Bootstrap starting.
I20260812 09:40:30.257805 9559 tablet_bootstrap.cc:654] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:30.257859 9561 tablet_bootstrap.cc:492] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea: Bootstrap starting.
I20260812 09:40:30.258806 9559 log.cc:824] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:30.259069 9560 tablet_bootstrap.cc:654] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:30.259898 9560 log.cc:824] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:30.260222 9561 tablet_bootstrap.cc:654] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:30.260937 9561 log.cc:824] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:30.261029 9559 tablet_bootstrap.cc:492] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: No bootstrap required, opened a new log
I20260812 09:40:30.261237 9559 ts_tablet_manager.cc:1397] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260812 09:40:30.262611 9561 tablet_bootstrap.cc:492] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea: No bootstrap required, opened a new log
I20260812 09:40:30.262823 9561 ts_tablet_manager.cc:1397] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 09:40:30.264384 9560 tablet_bootstrap.cc:492] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6: No bootstrap required, opened a new log
I20260812 09:40:30.264556 9560 ts_tablet_manager.cc:1397] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260812 09:40:30.265100 9559 raft_consensus.cc:348] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.265413 9559 raft_consensus.cc:374] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:30.265475 9559 raft_consensus.cc:729] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e04894e3bd484d828972d8f14dfd59, State: Initialized, Role: FOLLOWER
I20260812 09:40:30.265935 9559 consensus_queue.cc:260] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.266608 9539 heartbeater.cc:503] Master 127.3.147.62:41461 was elected leader, sending a full tablet report...
I20260812 09:40:30.266932 9559 ts_tablet_manager.cc:1428] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:40:30.267810 9561 raft_consensus.cc:348] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.268030 9561 raft_consensus.cc:374] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:30.268078 9561 raft_consensus.cc:729] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a39ff32af1b416d9a05b33bb6876eea, State: Initialized, Role: FOLLOWER
I20260812 09:40:30.268507 9561 consensus_queue.cc:260] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.269129 9234 heartbeater.cc:503] Master 127.3.147.62:41461 was elected leader, sending a full tablet report...
I20260812 09:40:30.268975 9560 raft_consensus.cc:348] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.269213 9560 raft_consensus.cc:374] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:30.269259 9560 raft_consensus.cc:729] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da810f3a10284ac885513631055427b6, State: Initialized, Role: FOLLOWER
I20260812 09:40:30.269447 9561 ts_tablet_manager.cc:1428] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260812 09:40:30.269668 9560 consensus_queue.cc:260] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.270311 9390 heartbeater.cc:503] Master 127.3.147.62:41461 was elected leader, sending a full tablet report...
I20260812 09:40:30.270677 9560 ts_tablet_manager.cc:1428] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:40:30.274073 9567 raft_consensus.cc:482] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:30.274245 9567 raft_consensus.cc:504] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.275174 9567 leader_election.cc:288] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84e04894e3bd484d828972d8f14dfd59 (127.3.147.3:42253), 1a39ff32af1b416d9a05b33bb6876eea (127.3.147.1:39267)
I20260812 09:40:30.277141 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.275472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:33347 (local address 127.3.147.3:42253)
0812 09:40:30.275580 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.275585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.275664 (+ 79us) server_negotiation.cc:407] Connection header received
0812 09:40:30.275773 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.275776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.275805 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.275901 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.276701 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.276704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.276707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.276719 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.276738 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.276748 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.276847 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.276967 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.277008 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 09:40:30.277127 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.275437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42253 (local address 127.3.147.2:33347)
0812 09:40:30.275596 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.275611 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:30.275697 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:30.275915 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:30.275917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:30.275931 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:30.276504 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:40:30.276507 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:30.276520 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:40:30.276556 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:30.276848 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:30.276851 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:30.276855 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 09:40:30.276954 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 09:40:30.276994 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 09:40:30.277228 9558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.275512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:45499 (local address 127.3.147.1:39267)
0812 09:40:30.275631 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.275635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.275891 (+ 256us) server_negotiation.cc:407] Connection header received
0812 09:40:30.275998 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.276002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.276033 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.276127 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.276845 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.276849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.276852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.276864 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.276886 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.276899 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.276983 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.277072 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.277107 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 09:40:30.277310 9569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.275463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:39267 (local address 127.3.147.2:45499)
0812 09:40:30.275799 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.275810 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:30.275919 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:30.276165 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:30.276168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:30.276183 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:30.276694 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:40:30.276696 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:30.276706 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:40:30.276732 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:30.276990 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:30.276993 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:30.276996 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 09:40:30.277074 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 09:40:30.277106 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":102,"threads_started":1}
I20260812 09:40:30.277820 9488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7ddd2ce9f044319d7b9a665409a452" candidate_uuid: "da810f3a10284ac885513631055427b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e04894e3bd484d828972d8f14dfd59" is_pre_election: true
I20260812 09:40:30.277828 9182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7ddd2ce9f044319d7b9a665409a452" candidate_uuid: "da810f3a10284ac885513631055427b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a39ff32af1b416d9a05b33bb6876eea" is_pre_election: true
I20260812 09:40:30.278095 9182 raft_consensus.cc:2454] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da810f3a10284ac885513631055427b6 in term 0.
I20260812 09:40:30.278084 9488 raft_consensus.cc:2454] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da810f3a10284ac885513631055427b6 in term 0.
I20260812 09:40:30.278571 9254 leader_election.cc:302] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e04894e3bd484d828972d8f14dfd59, da810f3a10284ac885513631055427b6; no voters:
I20260812 09:40:30.278851 9567 raft_consensus.cc:2781] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:30.278941 9567 raft_consensus.cc:482] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:30.278987 9567 raft_consensus.cc:3037] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:30.307526 9567 raft_consensus.cc:504] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.308048 9567 leader_election.cc:288] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [CANDIDATE]: Term 1 election: Requested vote from peers 84e04894e3bd484d828972d8f14dfd59 (127.3.147.3:42253), 1a39ff32af1b416d9a05b33bb6876eea (127.3.147.1:39267)
I20260812 09:40:30.308518 9488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7ddd2ce9f044319d7b9a665409a452" candidate_uuid: "da810f3a10284ac885513631055427b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e04894e3bd484d828972d8f14dfd59"
I20260812 09:40:30.308687 9488 raft_consensus.cc:3037] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:30.308739 9182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7ddd2ce9f044319d7b9a665409a452" candidate_uuid: "da810f3a10284ac885513631055427b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a39ff32af1b416d9a05b33bb6876eea"
I20260812 09:40:30.308882 9182 raft_consensus.cc:3037] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:30.310189 9182 raft_consensus.cc:2454] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da810f3a10284ac885513631055427b6 in term 1.
I20260812 09:40:30.310189 9488 raft_consensus.cc:2454] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da810f3a10284ac885513631055427b6 in term 1.
I20260812 09:40:30.310580 9255 leader_election.cc:302] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a39ff32af1b416d9a05b33bb6876eea, da810f3a10284ac885513631055427b6; no voters:
I20260812 09:40:30.310815 9567 raft_consensus.cc:2781] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:30.311115 9567 raft_consensus.cc:686] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 1 LEADER]: Becoming Leader. State: Replica: da810f3a10284ac885513631055427b6, State: Running, Role: LEADER
I20260812 09:40:30.311527 9567 consensus_queue.cc:237] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } }
I20260812 09:40:30.315399 9010 catalog_manager.cc:5495] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to da810f3a10284ac885513631055427b6 (127.3.147.2). New cstate: current_term: 1 leader_uuid: "da810f3a10284ac885513631055427b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da810f3a10284ac885513631055427b6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 } health_report { overall_health: UNKNOWN } } }
W20260812 09:40:30.329540 9235 tablet.cc:2369] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:40:30.335048 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.330075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.3.147.2:43741)
0812 09:40:30.330276 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.330280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.330293 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:30.330362 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.330364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.330416 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.330490 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.334639 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.334643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.334646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.334660 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.334701 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.334710 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.334791 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.334889 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.334928 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260812 09:40:30.341128 9182 raft_consensus.cc:1261] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [term 1 FOLLOWER]: Refusing update from remote peer da810f3a10284ac885513631055427b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:30.341167 9488 raft_consensus.cc:1261] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 1 FOLLOWER]: Refusing update from remote peer da810f3a10284ac885513631055427b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:30.341863 9570 consensus_queue.cc:1035] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a39ff32af1b416d9a05b33bb6876eea" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 39267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:30.342073 9567 consensus_queue.cc:1035] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e04894e3bd484d828972d8f14dfd59" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:30.352468 9575 mvcc.cc:205] Tried to move back new op lower bound from 7317617173869539328 to 7317617173759578112. Current Snapshot: MvccSnapshot[applied={T|T < 7317617173869539328 or (T in {7317617173869539328})}]
I20260812 09:40:30.353458 9576 mvcc.cc:205] Tried to move back new op lower bound from 7317617173869539328 to 7317617173759578112. Current Snapshot: MvccSnapshot[applied={T|T < 7317617173869539328}]
I20260812 09:40:30.358111 9009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:30.358254 9009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:30.360572 9009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d79ea4e8b91345afadd65d3c0e780808: Sending DeleteTablet for 3 replicas of tablet 6b7ddd2ce9f044319d7b9a665409a452
I20260812 09:40:30.361326 9463 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b7ddd2ce9f044319d7b9a665409a452 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:30 UTC) from {username='slave'} at 127.0.0.1:58298
I20260812 09:40:30.361406 9312 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b7ddd2ce9f044319d7b9a665409a452 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:30 UTC) from {username='slave'} at 127.0.0.1:39806
I20260812 09:40:30.361842 9587 tablet_replica.cc:330] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6: stopping tablet replica
I20260812 09:40:30.361769 9158 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b7ddd2ce9f044319d7b9a665409a452 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:30 UTC) from {username='slave'} at 127.0.0.1:58968
I20260812 09:40:30.362112 9587 raft_consensus.cc:2229] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:30.362442 9587 raft_consensus.cc:2258] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:30.362715 9589 tablet_replica.cc:330] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea: stopping tablet replica
I20260812 09:40:30.362782 9588 tablet_replica.cc:330] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: stopping tablet replica
I20260812 09:40:30.362998 9588 raft_consensus.cc:2229] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:30.362999 9589 raft_consensus.cc:2229] T 6b7ddd2ce9f044319d7b9a665409a452 P 1a39ff32af1b416d9a05b33bb6876eea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:30.363237 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 9083
I20260812 09:40:30.363286 9588 raft_consensus.cc:2258] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:30.364722 9587 ts_tablet_manager.cc:1905] T 6b7ddd2ce9f044319d7b9a665409a452 P da810f3a10284ac885513631055427b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:30.367501 9588 ts_tablet_manager.cc:1905] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 09:40:30.375759 8993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:39267 (error 108)
I20260812 09:40:30.375974 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 9237
I20260812 09:40:30.378669 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.378249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:39267 (local address 127.0.0.1:58970)
0812 09:40:30.378446 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.378551 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:39267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 09:40:30.378850 8993 catalog_manager.cc:4584] TS 1a39ff32af1b416d9a05b33bb6876eea (127.3.147.1:39267): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b7ddd2ce9f044319d7b9a665409a452: Network error: Client connection negotiation failed: client connection to 127.3.147.1:39267: connect: Connection refused (error 111)
I20260812 09:40:30.402526 9588 ts_tablet_manager.cc:1918] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:30.402716 9588 log.cc:1196] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/6b7ddd2ce9f044319d7b9a665409a452
I20260812 09:40:30.403115 9588 ts_tablet_manager.cc:1939] T 6b7ddd2ce9f044319d7b9a665409a452 P 84e04894e3bd484d828972d8f14dfd59: Deleting consensus metadata
I20260812 09:40:30.404858 8991 catalog_manager.cc:4841] TS 84e04894e3bd484d828972d8f14dfd59 (127.3.147.3:42253): tablet 6b7ddd2ce9f044319d7b9a665409a452 (table test-table [id=31277f0cd55a408bab844472cef90d54]) successfully deleted
I20260812 09:40:30.408749 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 9393
I20260812 09:40:30.409268 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.408908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:43741 (local address 127.0.0.1:39810)
0812 09:40:30.409111 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.409167 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.2:43741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 09:40:30.417953 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 8973
2026-08-12T09:40:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1135 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 09:40:30.442255 3660 mini_oidc.cc:152] Starting JWKS server
I20260812 09:40:30.444015 3660 webserver.cc:460] Webserver started at https://127.0.0.1:41815/ using document root <none> and password file <none>
I20260812 09:40:30.444362 3660 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 09:40:30.444693 3660 webserver.cc:460] Webserver started at http://127.0.0.1:35771/ using document root <none> and password file <none>
2026-08-12T09:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:30Z Disabled control of system clock
I20260812 09:40:30.454540 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:42351
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:39061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:42351
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41815/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/testchainca.cert with env {}
W20260812 09:40:30.564937 9599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:30.565188 9599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:30.565227 9599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:30.569252 9599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:30.569350 9599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:30.569380 9599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:30.569399 9599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:30.569415 9599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 09:40:30.569432 9599 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 09:40:30.569444 9599 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41815/jwks.json
I20260812 09:40:30.574728 9599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:42351
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:42351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41815/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:30.575688 9599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:30.576716 9599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:30.585029 9608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:30.585043 9605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:30.585173 9599 server_base.cc:1034] running on GCE node
W20260812 09:40:30.585029 9604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:30.585879 9599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:30.586927 9599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:30.588120 9599 hybrid_clock.cc:648] HybridClock initialized: now 1786527630588108 us; error 33 us; skew 500 ppm
I20260812 09:40:30.589946 9599 webserver.cc:460] Webserver started at http://127.3.147.62:37819/ using document root <none> and password file <none>
I20260812 09:40:30.590540 9599 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:30.590649 9599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:30.590876 9599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:30.593001 9599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "3d03d0edb73c494baf1b9c326f7c0ee0"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "e77e294bca4542c2ff4915e77b78b0c2"
server_key_iv: "ffd14fc57787e3307813c903ad8def42"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.593556 9599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "3d03d0edb73c494baf1b9c326f7c0ee0"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "e77e294bca4542c2ff4915e77b78b0c2"
server_key_iv: "ffd14fc57787e3307813c903ad8def42"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.597126 9599 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:40:30.599392 9614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:30.600565 9599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:30.600710 9599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "3d03d0edb73c494baf1b9c326f7c0ee0"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "e77e294bca4542c2ff4915e77b78b0c2"
server_key_iv: "ffd14fc57787e3307813c903ad8def42"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.600821 9599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:30.618352 9599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:30.619052 9599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:30.619257 9599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:30.626168 9599 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:42351
I20260812 09:40:30.626175 9677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:42351 every 8 connection(s)
I20260812 09:40:30.627086 9599 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:30.630292 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:30.630314 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 9599
I20260812 09:40:30.630456 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:30.630757 3660 external_mini_cluster.cc:1428] Setting key cd540361e06f68e8d5633fcd51529ae8
I20260812 09:40:30.637938 9678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0: Bootstrap starting.
I20260812 09:40:30.640717 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.632024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50144 (local address 127.3.147.62:42351)
0812 09:40:30.632687 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.632703 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.632738 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:40:30.633696 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.633715 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.634105 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.634386 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.634744 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:30.635664 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:30.636961 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:30.637415 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:30.637533 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:30.638491 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.638495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.638507 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.638968 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.639046 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.639078 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.639187 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.639361 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.639652 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":193,"threads_started":1}
I20260812 09:40:30.640759 9678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:30.641777 9678 log.cc:824] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:30.643673 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:40:30.644883 9678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0: No bootstrap required, opened a new log
I20260812 09:40:30.649538 9678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d03d0edb73c494baf1b9c326f7c0ee0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42351 } }
I20260812 09:40:30.649793 9678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:30.649864 9678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d03d0edb73c494baf1b9c326f7c0ee0, State: Initialized, Role: FOLLOWER
I20260812 09:40:30.650306 9678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d03d0edb73c494baf1b9c326f7c0ee0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42351 } }
I20260812 09:40:30.650458 9678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:30.650525 9678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:30.650622 9678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:30.651629 9678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d03d0edb73c494baf1b9c326f7c0ee0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42351 } }
I20260812 09:40:30.652010 9678 leader_election.cc:302] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d03d0edb73c494baf1b9c326f7c0ee0; no voters:
I20260812 09:40:30.652443 9678 leader_election.cc:288] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:30.652544 9683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:30.652815 9683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [term 1 LEADER]: Becoming Leader. State: Replica: 3d03d0edb73c494baf1b9c326f7c0ee0, State: Running, Role: LEADER
I20260812 09:40:30.653118 9683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d03d0edb73c494baf1b9c326f7c0ee0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42351 } }
I20260812 09:40:30.653716 9678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:30.654838 9684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d03d0edb73c494baf1b9c326f7c0ee0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d03d0edb73c494baf1b9c326f7c0ee0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42351 } } }
I20260812 09:40:30.654994 9684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:30.654837 9685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d03d0edb73c494baf1b9c326f7c0ee0. Latest consensus state: current_term: 1 leader_uuid: "3d03d0edb73c494baf1b9c326f7c0ee0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d03d0edb73c494baf1b9c326f7c0ee0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42351 } } }
I20260812 09:40:30.655145 9685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:30.655503 9693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:30.658280 9693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:30.663932 9693 catalog_manager.cc:1347] Generated new cluster ID: 6b878513bcb845ae948256e47c844451
I20260812 09:40:30.664018 9693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:30.693627 9693 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:30.694604 9693 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:30.700311 9693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d03d0edb73c494baf1b9c326f7c0ee0: Generated new TSK 0
I20260812 09:40:30.700980 9693 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 09:40:30.736021 9710 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 09:40:30.736264 9710 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 09:40:30.736840 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.717665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50150 (local address 127.3.147.62:42351)
0812 09:40:30.717810 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.717816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.717958 (+ 142us) server_negotiation.cc:407] Connection header received
0812 09:40:30.718165 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.718169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.718233 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.718355 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0812 09:40:30.718608 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:30.719381 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:30.720294 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:30.720647 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:30.720763 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:30.721081 (+ 318us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 09:40:30.736397 (+ 15316us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 09:40:30.736695 (+ 298us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50150: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 09:40:30.736927 9681 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50150: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 09:40:30.738265 9709 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:30.717533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42351 (local address 127.0.0.1:50150)
0812 09:40:30.717859 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:30.717872 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:30.717979 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:30.718364 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:30.718367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:30.718374 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0812 09:40:30.718480 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:30.718485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:30.719493 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:30.719495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:30.720114 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:30.720121 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:30.720765 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:30.720768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:30.720838 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:30.720879 (+ 41us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 09:40:30.736662 (+ 15783us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 09:40:30.736875 (+ 213us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.147.62:42351: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":112,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.147.62:42351: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 09:40:30.739298 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 9599
2026-08-12T09:40:30Z chronyd exiting
I20260812 09:40:30.781021 3660 test_util.cc:175] -----------------------------------------------
I20260812 09:40:30.781136 3660 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786527603645036-3660-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (340 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 09:40:30.782296 3660 mini_oidc.cc:152] Starting JWKS server
I20260812 09:40:30.783149 3660 webserver.cc:460] Webserver started at https://127.0.0.1:37317/ using document root <none> and password file <none>
I20260812 09:40:30.783282 3660 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 09:40:30.783548 3660 webserver.cc:460] Webserver started at http://127.0.0.1:43671/ using document root <none> and password file <none>
2026-08-12T09:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:30Z Disabled control of system clock
I20260812 09:40:30.792832 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44991
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:33433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:44991
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37317/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/testchainca.cert with env {}
W20260812 09:40:30.902530 9716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:30.902774 9716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:30.902812 9716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:30.906798 9716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:30.906849 9716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:30.906864 9716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:30.906880 9716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:30.906932 9716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 09:40:30.906975 9716 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 09:40:30.906997 9716 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37317/jwks.json
I20260812 09:40:30.912307 9716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:44991
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37317/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:30.913282 9716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:30.914263 9716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:30.922510 9722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:30.922640 9716 server_base.cc:1034] running on GCE node
W20260812 09:40:30.922572 9726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:30.922766 9723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:30.923385 9716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:30.924564 9716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:30.925776 9716 hybrid_clock.cc:648] HybridClock initialized: now 1786527630925750 us; error 48 us; skew 500 ppm
I20260812 09:40:30.927572 9716 webserver.cc:460] Webserver started at http://127.3.147.62:33659/ using document root <none> and password file <none>
I20260812 09:40:30.928249 9716 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:30.928350 9716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:30.928583 9716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:30.930614 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "0daec8d0c506468080f566269a5e2e95"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "c843fec5457aae6b9c04dd58036193c8"
server_key_iv: "d93496e3e9433602c716079a44a07a96"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.931195 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "0daec8d0c506468080f566269a5e2e95"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "c843fec5457aae6b9c04dd58036193c8"
server_key_iv: "d93496e3e9433602c716079a44a07a96"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.935014 9716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 09:40:30.937506 9735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:30.938763 9716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:30.938933 9716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "0daec8d0c506468080f566269a5e2e95"
format_stamp: "Formatted at 2026-08-12 09:40:30 on dist-test-slave-cbbz"
server_key: "c843fec5457aae6b9c04dd58036193c8"
server_key_iv: "d93496e3e9433602c716079a44a07a96"
server_key_version: "encryptionkey@0"
I20260812 09:40:30.939069 9716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:30.956518 9716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:30.957266 9716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:30.957489 9716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:30.964857 9716 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:44991
I20260812 09:40:30.964880 9798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:44991 every 8 connection(s)
I20260812 09:40:30.965844 9716 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:30.968206 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 9716
I20260812 09:40:30.968353 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:30.968696 3660 external_mini_cluster.cc:1428] Setting key e269d4ef6f508441b62ef772294bb9e2
I20260812 09:40:30.969125 9799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:30.976318 9799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95: Bootstrap starting.
I20260812 09:40:30.979051 9799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:30.979410 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:30.970188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.3.147.62:44991)
0812 09:40:30.970891 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:30.970912 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:30.970954 (+ 42us) server_negotiation.cc:407] Connection header received
0812 09:40:30.971865 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:30.971891 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:30.972398 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:30.972763 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:30.973202 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:30.974612 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:30.975538 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:30.976242 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:30.976411 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:30.977328 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:30.977334 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:30.977349 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:30.977872 (+ 523us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:30.977944 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:30.977970 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:30.978068 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:30.978207 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 09:40:30.978467 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260812 09:40:30.979904 9799 log.cc:824] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:30.981954 9799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95: No bootstrap required, opened a new log
I20260812 09:40:30.982182 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:40:30.985791 9799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0daec8d0c506468080f566269a5e2e95" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44991 } }
I20260812 09:40:30.985991 9799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:30.986025 9799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0daec8d0c506468080f566269a5e2e95, State: Initialized, Role: FOLLOWER
I20260812 09:40:30.986467 9799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0daec8d0c506468080f566269a5e2e95" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44991 } }
I20260812 09:40:30.986618 9799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:30.986702 9799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:30.986830 9799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:30.987700 9799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0daec8d0c506468080f566269a5e2e95" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44991 } }
I20260812 09:40:30.988037 9799 leader_election.cc:302] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0daec8d0c506468080f566269a5e2e95; no voters:
I20260812 09:40:30.988363 9799 leader_election.cc:288] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:30.988512 9806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:30.988747 9806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [term 1 LEADER]: Becoming Leader. State: Replica: 0daec8d0c506468080f566269a5e2e95, State: Running, Role: LEADER
I20260812 09:40:30.989063 9806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0daec8d0c506468080f566269a5e2e95" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44991 } }
I20260812 09:40:30.989547 9799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:30.990767 9807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0daec8d0c506468080f566269a5e2e95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0daec8d0c506468080f566269a5e2e95" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44991 } } }
I20260812 09:40:30.990760 9808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0daec8d0c506468080f566269a5e2e95. Latest consensus state: current_term: 1 leader_uuid: "0daec8d0c506468080f566269a5e2e95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0daec8d0c506468080f566269a5e2e95" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44991 } } }
I20260812 09:40:30.990896 9807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:30.990942 9808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:30.991328 9816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:30.994453 9816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:31.000104 9816 catalog_manager.cc:1347] Generated new cluster ID: 232cac95ea0a4988b9651ce0f525f80c
I20260812 09:40:31.000222 9816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:31.015651 9816 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:31.016633 9816 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:31.028642 9816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0daec8d0c506468080f566269a5e2e95: Generated new TSK 0
I20260812 09:40:31.029358 9816 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 09:40:31.063241 9834 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 09:40:31.063339 9834 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 09:40:31.063984 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:31.040427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.3.147.62:44991)
0812 09:40:31.040586 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:31.040592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:31.040811 (+ 219us) server_negotiation.cc:407] Connection header received
0812 09:40:31.040957 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:31.040960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:31.041007 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:31.041111 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0812 09:40:31.041365 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.041898 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.043171 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.043383 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.043530 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:31.043852 (+ 322us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 09:40:31.063623 (+ 19771us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 09:40:31.063868 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51002: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 09:40:31.064078 9802 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51002: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 09:40:31.064260 9833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:31.040385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44991 (local address 127.0.0.1:51002)
0812 09:40:31.040703 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:31.040719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:31.040845 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:31.041127 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:31.041130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:31.041138 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 09:40:31.041250 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:31.041256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.042014 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.042018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:31.043056 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:31.043067 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.043556 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.043560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:31.043653 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:31.043682 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 09:40:31.063857 (+ 20175us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 09:40:31.064050 (+ 193us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.147.62:44991: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37317/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":127,"threads_started":1}
I20260812 09:40:31.065088 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 9716
2026-08-12T09:40:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (326 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 09:40:31.108335 3660 mini_oidc.cc:152] Starting JWKS server
I20260812 09:40:31.109212 3660 webserver.cc:460] Webserver started at https://127.0.0.1:34163/ using document root <none> and password file <none>
I20260812 09:40:31.109329 3660 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 09:40:31.109608 3660 webserver.cc:460] Webserver started at http://127.0.0.1:41303/ using document root <none> and password file <none>
2026-08-12T09:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:31Z Disabled control of system clock
I20260812 09:40:31.119510 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41373
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:32873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:41373
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34163/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 09:40:31.230939 9841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:31.231190 9841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:31.231230 9841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:31.235293 9841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:31.235344 9841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:31.235359 9841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:31.235375 9841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:31.235391 9841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 09:40:31.235406 9841 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 09:40:31.235425 9841 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34163/jwks.json
I20260812 09:40:31.239997 9841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:32873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:41373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34163/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:31.240994 9841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:31.242345 9841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:31.250619 9850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:31.250661 9848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:31.250802 9846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:31.250895 9841 server_base.cc:1034] running on GCE node
I20260812 09:40:31.251541 9841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:31.252647 9841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:31.253850 9841 hybrid_clock.cc:648] HybridClock initialized: now 1786527631253820 us; error 39 us; skew 500 ppm
I20260812 09:40:31.255805 9841 webserver.cc:460] Webserver started at http://127.3.147.62:41743/ using document root <none> and password file <none>
I20260812 09:40:31.256455 9841 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:31.256525 9841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:31.256783 9841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:31.258881 9841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "8d742bf794074b75afd7eba9e1242ec0"
format_stamp: "Formatted at 2026-08-12 09:40:31 on dist-test-slave-cbbz"
server_key: "d4304269874b937c91bec9614f39bf44"
server_key_iv: "56e56b8211435accc83a5f5b945d37d5"
server_key_version: "encryptionkey@0"
I20260812 09:40:31.259482 9841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "8d742bf794074b75afd7eba9e1242ec0"
format_stamp: "Formatted at 2026-08-12 09:40:31 on dist-test-slave-cbbz"
server_key: "d4304269874b937c91bec9614f39bf44"
server_key_iv: "56e56b8211435accc83a5f5b945d37d5"
server_key_version: "encryptionkey@0"
I20260812 09:40:31.263312 9841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 09:40:31.265872 9858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:31.267019 9841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:31.267164 9841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "8d742bf794074b75afd7eba9e1242ec0"
format_stamp: "Formatted at 2026-08-12 09:40:31 on dist-test-slave-cbbz"
server_key: "d4304269874b937c91bec9614f39bf44"
server_key_iv: "56e56b8211435accc83a5f5b945d37d5"
server_key_version: "encryptionkey@0"
I20260812 09:40:31.267273 9841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:31.289737 9841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:31.290398 9841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:31.290618 9841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:31.298121 9841 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:41373
I20260812 09:40:31.298120 9921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:41373 every 8 connection(s)
I20260812 09:40:31.299103 9841 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:31.302287 9922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:31.305436 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 9841
I20260812 09:40:31.305548 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:31.305850 3660 external_mini_cluster.cc:1428] Setting key fe1a6843ad61b956bb94e34b6513956e
I20260812 09:40:31.308423 9922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0: Bootstrap starting.
I20260812 09:40:31.310801 9922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:31.311529 9922 log.cc:824] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:31.313839 9922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0: No bootstrap required, opened a new log
I20260812 09:40:31.315860 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:31.307395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.3.147.62:41373)
0812 09:40:31.308062 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:31.308081 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:31.308119 (+ 38us) server_negotiation.cc:407] Connection header received
0812 09:40:31.308889 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:31.308907 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:31.309327 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:31.309670 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:31.310095 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.311114 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.312080 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.312752 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.312940 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:31.313850 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:31.313860 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:31.313878 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:31.314354 (+ 476us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:31.314418 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:31.314444 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:31.314541 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:31.314685 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 09:40:31.314941 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":136,"threads_started":1}
I20260812 09:40:31.318135 9922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d742bf794074b75afd7eba9e1242ec0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41373 } }
I20260812 09:40:31.318393 9922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:31.318451 9922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d742bf794074b75afd7eba9e1242ec0, State: Initialized, Role: FOLLOWER
I20260812 09:40:31.318831 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:40:31.318886 9922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d742bf794074b75afd7eba9e1242ec0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41373 } }
I20260812 09:40:31.319077 9922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:31.319144 9922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:31.319238 9922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:31.320281 9922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d742bf794074b75afd7eba9e1242ec0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41373 } }
I20260812 09:40:31.320618 9922 leader_election.cc:302] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d742bf794074b75afd7eba9e1242ec0; no voters:
I20260812 09:40:31.320911 9922 leader_election.cc:288] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:31.321053 9929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:31.321309 9929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [term 1 LEADER]: Becoming Leader. State: Replica: 8d742bf794074b75afd7eba9e1242ec0, State: Running, Role: LEADER
I20260812 09:40:31.321604 9929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d742bf794074b75afd7eba9e1242ec0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41373 } }
I20260812 09:40:31.322081 9922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:31.323191 9933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d742bf794074b75afd7eba9e1242ec0. Latest consensus state: current_term: 1 leader_uuid: "8d742bf794074b75afd7eba9e1242ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d742bf794074b75afd7eba9e1242ec0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41373 } } }
I20260812 09:40:31.323323 9933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:31.323608 9930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d742bf794074b75afd7eba9e1242ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d742bf794074b75afd7eba9e1242ec0" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41373 } } }
I20260812 09:40:31.323704 9940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:31.323727 9930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:31.326916 9940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:31.332676 9940 catalog_manager.cc:1347] Generated new cluster ID: a79da878215c40d1b522294f6513507b
I20260812 09:40:31.332751 9940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:31.351285 9940 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:31.352344 9940 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:31.360435 9940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d742bf794074b75afd7eba9e1242ec0: Generated new TSK 0
I20260812 09:40:31.361131 9940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:40:31.374897 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:31.369741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35404 (local address 127.3.147.62:41373)
0812 09:40:31.369867 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:31.369872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:31.370079 (+ 207us) server_negotiation.cc:407] Connection header received
0812 09:40:31.370276 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:31.370279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:31.370327 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:31.370437 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0812 09:40:31.370700 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.371383 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.372471 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.372709 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.372824 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:31.373149 (+ 325us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 09:40:31.374547 (+ 1398us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 09:40:31.374770 (+ 223us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35404: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 09:40:31.374959 9925 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35404: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 09:40:31.376868 9959 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:31.369650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41373 (local address 127.0.0.1:35404)
0812 09:40:31.369986 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:31.369998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:31.370133 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:31.370452 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:31.370456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:31.370464 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 09:40:31.370583 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:31.370588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.371521 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.371525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:31.372351 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:31.372361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.372836 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.372840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:31.372942 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:31.372964 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 09:40:31.374727 (+ 1763us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 09:40:31.376692 (+ 1965us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.147.62:41373: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":1757,"thread_start_us":100,"threads_started":1}
I20260812 09:40:31.377218 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 9841
E20260812 09:40:31.386806 9960 webserver.cc:533] Webserver: sslize failed:
E20260812 09:40:31.386900 9960 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T09:40:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (311 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T09:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:31Z Disabled control of system clock
I20260812 09:40:31.428996 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:45917
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:45917
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/testchainca.cert with env {}
W20260812 09:40:31.536705 9965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:31.536978 9965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:31.537048 9965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:31.541006 9965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:31.541083 9965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:31.541111 9965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:31.541175 9965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:31.541213 9965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 09:40:31.541247 9965 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 09:40:31.541280 9965 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 09:40:31.546406 9965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:45917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:31.547387 9965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:31.548404 9965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:31.556677 9970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:31.556656 9974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:31.556682 9971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:31.556818 9965 server_base.cc:1034] running on GCE node
I20260812 09:40:31.557710 9965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:31.558728 9965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:31.559916 9965 hybrid_clock.cc:648] HybridClock initialized: now 1786527631559896 us; error 30 us; skew 500 ppm
I20260812 09:40:31.561697 9965 webserver.cc:460] Webserver started at http://127.3.147.62:43667/ using document root <none> and password file <none>
I20260812 09:40:31.562270 9965 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:31.562367 9965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:31.562590 9965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:31.564632 9965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "7ff9854c11af4ce1948944874038ed1d"
format_stamp: "Formatted at 2026-08-12 09:40:31 on dist-test-slave-cbbz"
server_key: "5c80344643e131d489ca2e7940786090"
server_key_iv: "7e327cc1d4da2f74a4ef0596e2858602"
server_key_version: "encryptionkey@0"
I20260812 09:40:31.565197 9965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "7ff9854c11af4ce1948944874038ed1d"
format_stamp: "Formatted at 2026-08-12 09:40:31 on dist-test-slave-cbbz"
server_key: "5c80344643e131d489ca2e7940786090"
server_key_iv: "7e327cc1d4da2f74a4ef0596e2858602"
server_key_version: "encryptionkey@0"
I20260812 09:40:31.568660 9965 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:40:31.571092 9980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:31.572439 9965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:31.572609 9965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "7ff9854c11af4ce1948944874038ed1d"
format_stamp: "Formatted at 2026-08-12 09:40:31 on dist-test-slave-cbbz"
server_key: "5c80344643e131d489ca2e7940786090"
server_key_iv: "7e327cc1d4da2f74a4ef0596e2858602"
server_key_version: "encryptionkey@0"
I20260812 09:40:31.572731 9965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:31.598143 9965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:31.598867 9965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:31.599107 9965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:31.606769 9965 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:45917
I20260812 09:40:31.606777 10038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:45917 every 8 connection(s)
I20260812 09:40:31.607759 9965 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:31.610978 10039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:31.614753 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 9965
I20260812 09:40:31.614934 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:31.615245 3660 external_mini_cluster.cc:1428] Setting key 76aa1e6c69cb1bfea3e004536a524aba
I20260812 09:40:31.616744 10039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d: Bootstrap starting.
I20260812 09:40:31.619072 10039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:31.619868 10039 log.cc:824] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:31.621721 10039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d: No bootstrap required, opened a new log
I20260812 09:40:31.624383 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:31.616654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35014 (local address 127.3.147.62:45917)
0812 09:40:31.617289 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:31.617305 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:31.617339 (+ 34us) server_negotiation.cc:407] Connection header received
0812 09:40:31.617971 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:31.617988 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:31.618371 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:31.618640 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:31.618971 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.620218 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.621087 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.621557 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.621747 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:31.622640 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:31.622645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:31.622657 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:40:31.623049 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:31.623097 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:40:31.623116 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:31.623234 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:31.623389 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 09:40:31.623641 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":144,"threads_started":1}
I20260812 09:40:31.625604 10039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff9854c11af4ce1948944874038ed1d" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45917 } }
I20260812 09:40:31.625871 10039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:31.625921 10039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ff9854c11af4ce1948944874038ed1d, State: Initialized, Role: FOLLOWER
I20260812 09:40:31.626400 10039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff9854c11af4ce1948944874038ed1d" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45917 } }
I20260812 09:40:31.626559 10039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:31.626646 10039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:31.626729 10039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:31.627293 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:40:31.627835 10039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff9854c11af4ce1948944874038ed1d" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45917 } }
I20260812 09:40:31.628207 10039 leader_election.cc:302] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ff9854c11af4ce1948944874038ed1d; no voters:
I20260812 09:40:31.628583 10039 leader_election.cc:288] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:31.628703 10045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:31.628958 10045 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [term 1 LEADER]: Becoming Leader. State: Replica: 7ff9854c11af4ce1948944874038ed1d, State: Running, Role: LEADER
I20260812 09:40:31.629249 10045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff9854c11af4ce1948944874038ed1d" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45917 } }
I20260812 09:40:31.629846 10039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:31.631160 10046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ff9854c11af4ce1948944874038ed1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff9854c11af4ce1948944874038ed1d" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45917 } } }
I20260812 09:40:31.631299 10046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:31.631645 10047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ff9854c11af4ce1948944874038ed1d. Latest consensus state: current_term: 1 leader_uuid: "7ff9854c11af4ce1948944874038ed1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff9854c11af4ce1948944874038ed1d" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45917 } } }
I20260812 09:40:31.631765 10047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:31.631982 10054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:31.634651 10054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:31.639627 10054 catalog_manager.cc:1347] Generated new cluster ID: f5c52937de084619a622e5aa828a4ced
I20260812 09:40:31.639688 10054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:31.671224 10054 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:31.672535 10054 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:31.680204 10054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ff9854c11af4ce1948944874038ed1d: Generated new TSK 0
I20260812 09:40:31.681017 10054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:40:31.722059 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:31.695212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35026 (local address 127.3.147.62:45917)
0812 09:40:31.695353 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:31.695358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:31.695519 (+ 161us) server_negotiation.cc:407] Connection header received
0812 09:40:31.695646 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:31.695650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:31.695707 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:31.695825 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0812 09:40:31.696034 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.696884 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.697722 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.698071 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.698206 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:31.698489 (+ 283us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 09:40:31.721548 (+ 23059us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 09:40:31.721906 (+ 358us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35026: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 09:40:31.722168 10043 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35026: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 09:40:31.722213 10074 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:31.695092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:45917 (local address 127.0.0.1:35026)
0812 09:40:31.695424 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:31.695436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:31.695542 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:31.695829 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:31.695831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:31.695838 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0812 09:40:31.695931 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:31.695936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.696996 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.696998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:31.697609 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:31.697616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:31.698223 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:31.698226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:31.698295 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:31.698316 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 09:40:31.721823 (+ 23507us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 09:40:31.722014 (+ 191us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.147.62:45917: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":128,"threads_started":1}
I20260812 09:40:31.723059 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 9965
2026-08-12T09:40:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (325 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (127 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (132 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:32 dist-test-slave-cbbz krb5kdc[10090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:32 dist-test-slave-cbbz krb5kdc[10090](info): set up 2 sockets
Aug 12 09:40:32 dist-test-slave-cbbz krb5kdc[10090](info): commencing operation
krb5kdc: starting...
W20260812 09:40:33.093292 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:33 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:33.221287 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41899
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:41899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:33.334673 10104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:33.334918 10104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:33.334957 10104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:33.339046 10104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:33.339150 10104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:33.339193 10104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:33.339215 10104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:33.339288 10104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:33.344437 10104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:41899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:41899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:33.345486 10104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:33.346501 10104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:33.354463 10113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:33.354502 10111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:33.354537 10110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:33.354883 10104 server_base.cc:1034] running on GCE node
I20260812 09:40:33.355398 10104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:33.356480 10104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:33.357654 10104 hybrid_clock.cc:648] HybridClock initialized: now 1786527633357635 us; error 30 us; skew 500 ppm
Aug 12 09:40:33 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527633, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:33.360438 10104 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:33.361285 10104 webserver.cc:460] Webserver started at http://127.3.147.62:36037/ using document root <none> and password file <none>
I20260812 09:40:33.361835 10104 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:33.361897 10104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:33.362136 10104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:33.364176 10104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "405b64e44ca447aaa10c2ecb6601a007"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "d02e629669507b66f93b93200f55d7e2"
server_key_iv: "b3ccf80b342b58ad376f0a7a511c927b"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.364743 10104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "405b64e44ca447aaa10c2ecb6601a007"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "d02e629669507b66f93b93200f55d7e2"
server_key_iv: "b3ccf80b342b58ad376f0a7a511c927b"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.368614 10104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 09:40:33.370918 10121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:33.372174 10104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:33.372316 10104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "405b64e44ca447aaa10c2ecb6601a007"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "d02e629669507b66f93b93200f55d7e2"
server_key_iv: "b3ccf80b342b58ad376f0a7a511c927b"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.372431 10104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:33.411069 10104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:33.414326 10104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:33.414538 10104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:33.421934 10104 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:41899
I20260812 09:40:33.421964 10179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:41899 every 8 connection(s)
I20260812 09:40:33.422873 10104 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:33.425639 10180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:33.427456 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 10104
I20260812 09:40:33.427577 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:33.427879 3660 external_mini_cluster.cc:1428] Setting key fa0448bc437a514cd311b90a257ffdc8
I20260812 09:40:33.431946 10180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007: Bootstrap starting.
I20260812 09:40:33.434193 10180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:33.435144 10180 log.cc:824] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:40:33 dist-test-slave-cbbz krb5kdc[10090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:33.437273 10180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007: No bootstrap required, opened a new log
I20260812 09:40:33.442163 10180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405b64e44ca447aaa10c2ecb6601a007" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41899 } }
I20260812 09:40:33.442345 10184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:33.429360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.3.147.62:41899)
0812 09:40:33.429789 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:33.429803 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:33.429838 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:40:33.430459 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:33.430475 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:33.430786 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:33.431109 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:33.431920 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.432964 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.433810 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.434257 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.434429 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:33.437158 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:33.437164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:33.437180 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:33.437209 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:33.439076 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:33.439654 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:33.439663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:33.439670 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:33.439841 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:33.440157 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:33.440160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:33.440162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:33.440671 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:33.440873 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:33.441075 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 09:40:33.441375 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":123,"threads_started":1}
I20260812 09:40:33.442437 10180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:33.442493 10180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405b64e44ca447aaa10c2ecb6601a007, State: Initialized, Role: FOLLOWER
I20260812 09:40:33.442941 10180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405b64e44ca447aaa10c2ecb6601a007" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41899 } }
I20260812 09:40:33.443094 10180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:33.443171 10180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:33.443266 10180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:33.444480 10180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405b64e44ca447aaa10c2ecb6601a007" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41899 } }
I20260812 09:40:33.444809 10180 leader_election.cc:302] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 405b64e44ca447aaa10c2ecb6601a007; no voters:
I20260812 09:40:33.445200 10186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:33.445340 10180 leader_election.cc:288] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:33.445441 10186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [term 1 LEADER]: Becoming Leader. State: Replica: 405b64e44ca447aaa10c2ecb6601a007, State: Running, Role: LEADER
I20260812 09:40:33.445753 10186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405b64e44ca447aaa10c2ecb6601a007" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41899 } }
I20260812 09:40:33.446672 10180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:33.447113 10188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 405b64e44ca447aaa10c2ecb6601a007. Latest consensus state: current_term: 1 leader_uuid: "405b64e44ca447aaa10c2ecb6601a007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405b64e44ca447aaa10c2ecb6601a007" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41899 } } }
I20260812 09:40:33.447237 10188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:33.447630 10191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:33.447584 10189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "405b64e44ca447aaa10c2ecb6601a007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405b64e44ca447aaa10c2ecb6601a007" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 41899 } } }
I20260812 09:40:33.448643 10189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:33.451047 10191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:33.456220 10191 catalog_manager.cc:1347] Generated new cluster ID: 9d2763b43d2845e9a41ac89d4a05c84a
I20260812 09:40:33.456282 10191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:33.481897 10191 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:33.483103 10191 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:33.489933 10191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 405b64e44ca447aaa10c2ecb6601a007: Generated new TSK 0
I20260812 09:40:33.490685 10191 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:33.527817 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41899
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:33.644809 10212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:33.645046 10212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:33.645124 10212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:33.649291 10212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:33.649343 10212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:33.649446 10212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:33.654683 10212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:33.655685 10212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:33.656694 10212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:33.665920 10218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:33.665970 10220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:33.666090 10212 server_base.cc:1034] running on GCE node
W20260812 09:40:33.665930 10217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:33.666735 10212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:33.667338 10212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:33.668501 10212 hybrid_clock.cc:648] HybridClock initialized: now 1786527633668474 us; error 40 us; skew 500 ppm
Aug 12 09:40:33 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527633, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:33.671128 10212 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:33.671917 10212 webserver.cc:460] Webserver started at http://127.3.147.1:43065/ using document root <none> and password file <none>
I20260812 09:40:33.672504 10212 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:33.672598 10212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:33.672828 10212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:33.674791 10212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "2dac4817231f4e169af3108bb4091c35"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "afae28ab453fabc0d795a9808d8cc2ad"
server_key_iv: "7cbcc60c29c7271b34f7f9cdf1a2d015"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.675359 10212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "2dac4817231f4e169af3108bb4091c35"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "afae28ab453fabc0d795a9808d8cc2ad"
server_key_iv: "7cbcc60c29c7271b34f7f9cdf1a2d015"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.679148 10212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:33.681560 10228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:33.682753 10212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:33.682890 10212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "2dac4817231f4e169af3108bb4091c35"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "afae28ab453fabc0d795a9808d8cc2ad"
server_key_iv: "7cbcc60c29c7271b34f7f9cdf1a2d015"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.683003 10212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:33.709447 10212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:33.712453 10212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:33.712678 10212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:33.713447 10212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:33.714599 10212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:33.714659 10212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:33.714738 10212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:33.714771 10212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:33.727578 10212 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:45141
I20260812 09:40:33.727599 10362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:45141 every 8 connection(s)
I20260812 09:40:33.728564 10212 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:33.735023 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 10212
I20260812 09:40:33.735144 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:33.735471 3660 external_mini_cluster.cc:1428] Setting key 858402816f1581eafdbf83aaa7a6e887
Aug 12 09:40:33 dist-test-slave-cbbz krb5kdc[10090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527633, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:33.743886 10184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:33.730427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:44039 (local address 127.3.147.62:41899)
0812 09:40:33.730574 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:33.730578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:33.731420 (+ 842us) server_negotiation.cc:407] Connection header received
0812 09:40:33.733168 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:33.733172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:33.733222 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:33.733298 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:33.734421 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.735087 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.736395 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.736729 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.736868 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:33.739683 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:33.739690 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:33.739696 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:33.739729 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:33.741559 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:33.742094 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:33.742101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:33.742106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:33.742207 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:33.742614 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:33.742617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:33.742619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:33.742855 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:33.742970 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:33.743504 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 09:40:33.743651 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 09:40:33.744540 10365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:33.730811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41899 (local address 127.3.147.1:44039)
0812 09:40:33.731258 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:33.731302 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:33.732963 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:33.733465 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:33.733472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:33.733822 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:33.734287 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:33.734300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.735270 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.735276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:33.736281 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:33.736291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.736918 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.736922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:33.737130 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:33.737739 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:33.737795 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:33.739506 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:33.741689 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:33.741694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:33.741705 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:33.741978 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:33.742341 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:33.742347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:33.742349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:33.742506 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:33.742967 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:33.742972 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:33.743224 (+ 252us) client_negotiation.cc:769] Sending connection context
0812 09:40:33.743391 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 09:40:33.743648 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260812 09:40:33.745810 10363 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41899
I20260812 09:40:33.746066 10363 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:33.746680 10363 heartbeater.cc:511] Master 127.3.147.62:41899 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:33.748420 10139 ts_manager.cc:194] Registered new tserver with Master: 2dac4817231f4e169af3108bb4091c35 (127.3.147.1:45141)
I20260812 09:40:33.749637 10139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:44039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:33.767319 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41899
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:33.880131 10368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:33.880470 10368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:33.880550 10368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:33.884687 10368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:33.884768 10368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:33.884855 10368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:33.890204 10368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:33.891268 10368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:33.892364 10368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:33.902069 10374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:33.902083 10377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:33.902110 10375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:33.902511 10368 server_base.cc:1034] running on GCE node
I20260812 09:40:33.902930 10368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:33.903589 10368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:33.904784 10368 hybrid_clock.cc:648] HybridClock initialized: now 1786527633904769 us; error 35 us; skew 500 ppm
Aug 12 09:40:33 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527633, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:33.907672 10368 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:33.908584 10368 webserver.cc:460] Webserver started at http://127.3.147.2:41917/ using document root <none> and password file <none>
I20260812 09:40:33.909137 10368 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:33.909246 10368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:33.909483 10368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:33.911468 10368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "7bd841d4c5194da6951d2a154b2d94a6"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "5083be3d9e58af6dd75711200d3e8f99"
server_key_iv: "e48e83275746d5c500c70e70e4d05a86"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.912036 10368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "7bd841d4c5194da6951d2a154b2d94a6"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "5083be3d9e58af6dd75711200d3e8f99"
server_key_iv: "e48e83275746d5c500c70e70e4d05a86"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.915803 10368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:33.918330 10387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:33.919549 10368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:33.919688 10368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "7bd841d4c5194da6951d2a154b2d94a6"
format_stamp: "Formatted at 2026-08-12 09:40:33 on dist-test-slave-cbbz"
server_key: "5083be3d9e58af6dd75711200d3e8f99"
server_key_iv: "e48e83275746d5c500c70e70e4d05a86"
server_key_version: "encryptionkey@0"
I20260812 09:40:33.919803 10368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:33.951157 10368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:33.954771 10368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:33.954995 10368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:33.955657 10368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:33.956586 10368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:33.956661 10368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:33.956736 10368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:33.956791 10368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:33.966012 10368 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46073
I20260812 09:40:33.966045 10515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46073 every 8 connection(s)
I20260812 09:40:33.966897 10368 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:33.973667 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 10368
I20260812 09:40:33.973783 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:33.974097 3660 external_mini_cluster.cc:1428] Setting key 7aa99417b4728547fd7d3b0a2714a5b3
Aug 12 09:40:33 dist-test-slave-cbbz krb5kdc[10090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527633, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:33.982010 10184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:33.968721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:51991 (local address 127.3.147.62:41899)
0812 09:40:33.968865 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:33.968869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:33.969723 (+ 854us) server_negotiation.cc:407] Connection header received
0812 09:40:33.971084 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:33.971087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:33.971151 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:33.971226 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:33.972510 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.973033 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.974611 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.974871 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.974992 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:33.978100 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:33.978103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:33.978105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:33.978129 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:33.979559 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:33.980257 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:33.980260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:33.980261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:33.980335 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:33.980787 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:33.980790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:33.980791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:33.980959 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:33.981032 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:33.981763 (+ 731us) server_negotiation.cc:299] Negotiation successful
0812 09:40:33.981891 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:40:33.983446 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:33.969074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41899 (local address 127.3.147.2:51991)
0812 09:40:33.969552 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:33.969598 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:33.970892 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:33.971393 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:33.971399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:33.971744 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:33.972367 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:33.972385 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.973162 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.973165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:33.974473 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:33.974484 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:33.975036 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:33.975041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:33.975210 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:33.975920 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:33.975941 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:33.977914 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:33.979679 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:33.979690 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:33.979708 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:33.980122 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:33.980437 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:33.980444 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:33.980449 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:33.980669 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:33.981083 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:33.981090 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:33.981473 (+ 383us) client_negotiation.cc:769] Sending connection context
0812 09:40:33.981712 (+ 239us) client_negotiation.cc:240] Negotiation successful
0812 09:40:33.982149 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
I20260812 09:40:33.984831 10516 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41899
I20260812 09:40:33.985085 10516 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:33.985580 10516 heartbeater.cc:511] Master 127.3.147.62:41899 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:40:33.986743 10139 ts_manager.cc:194] Registered new tserver with Master: 7bd841d4c5194da6951d2a154b2d94a6 (127.3.147.2:46073)
I20260812 09:40:33.987403 10139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:51991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:34.007627 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41899
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:34.119993 10522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:34.120348 10522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:34.120438 10522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:34.124580 10522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:34.124672 10522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:34.124763 10522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:34.130020 10522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:34.131074 10522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:34.132121 10522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:40:34.142028 10522 server_base.cc:1034] running on GCE node
W20260812 09:40:34.141975 10527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:34.141995 10532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:34.142099 10529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:34.142707 10522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:34.143385 10522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:34.144560 10522 hybrid_clock.cc:648] HybridClock initialized: now 1786527634144542 us; error 30 us; skew 500 ppm
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:34.147727 10522 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:34.148909 10522 webserver.cc:460] Webserver started at http://127.3.147.3:46703/ using document root <none> and password file <none>
I20260812 09:40:34.149669 10522 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:34.149765 10522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:34.150022 10522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:34.153033 10522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "b346b283db684e20a25a024791235cf5"
format_stamp: "Formatted at 2026-08-12 09:40:34 on dist-test-slave-cbbz"
server_key: "d8f961c27918ed575c2c881f1717e7a0"
server_key_iv: "34553536869c2c5365b13613b0989cf7"
server_key_version: "encryptionkey@0"
I20260812 09:40:34.153784 10522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "b346b283db684e20a25a024791235cf5"
format_stamp: "Formatted at 2026-08-12 09:40:34 on dist-test-slave-cbbz"
server_key: "d8f961c27918ed575c2c881f1717e7a0"
server_key_iv: "34553536869c2c5365b13613b0989cf7"
server_key_version: "encryptionkey@0"
I20260812 09:40:34.159025 10522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 09:40:34.162389 10541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:34.163960 10522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:34.164088 10522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "b346b283db684e20a25a024791235cf5"
format_stamp: "Formatted at 2026-08-12 09:40:34 on dist-test-slave-cbbz"
server_key: "d8f961c27918ed575c2c881f1717e7a0"
server_key_iv: "34553536869c2c5365b13613b0989cf7"
server_key_version: "encryptionkey@0"
I20260812 09:40:34.164239 10522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:34.197192 10522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:34.201093 10522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:34.201352 10522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:34.202077 10522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:34.203014 10522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:34.203073 10522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:34.203166 10522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:34.203228 10522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:34.214849 10522 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:45697
I20260812 09:40:34.214883 10675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:45697 every 8 connection(s)
I20260812 09:40:34.215910 10522 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:34.218971 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 10522
I20260812 09:40:34.219169 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:34.219537 3660 external_mini_cluster.cc:1428] Setting key f2d34be85332c77d7606a2353d3dcd8a
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:34.232487 10184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:34.218174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:53639 (local address 127.3.147.62:41899)
0812 09:40:34.218341 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:34.218346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:34.219426 (+ 1080us) server_negotiation.cc:407] Connection header received
0812 09:40:34.221518 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:34.221521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:34.221582 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:34.221678 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:34.223206 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.223763 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.225172 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.225431 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.225566 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:34.228683 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:34.228687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:34.228689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:34.228716 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:34.230293 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:34.230837 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:34.230840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:34.230842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:34.230933 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:34.231310 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:34.231313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:34.231315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:34.231503 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:34.231619 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:34.232183 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 09:40:34.232313 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 09:40:34.233484 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:34.218566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:41899 (local address 127.3.147.3:53639)
0812 09:40:34.219248 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:34.219311 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:34.221268 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:34.222027 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:34.222034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:34.222423 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:34.223042 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:34.223067 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.223925 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.223929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:34.225024 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:34.225036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.225601 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.225605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:34.225794 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:34.226502 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:34.226519 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:34.228486 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:34.230427 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:34.230433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:34.230454 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:34.230718 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:34.231031 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:34.231034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:34.231036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:34.231191 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:34.231624 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:34.231633 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:34.231897 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 09:40:34.232099 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 09:40:34.232433 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":187,"threads_started":1}
I20260812 09:40:34.234812 10676 heartbeater.cc:347] Connected to a master server at 127.3.147.62:41899
I20260812 09:40:34.235081 10676 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:34.235669 10676 heartbeater.cc:511] Master 127.3.147.62:41899 requested a full tablet report, sending...
I20260812 09:40:34.236804 10139 ts_manager.cc:194] Registered new tserver with Master: b346b283db684e20a25a024791235cf5 (127.3.147.3:45697)
I20260812 09:40:34.237396 10139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:53639
I20260812 09:40:34.246097 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/test-admin.keytab.
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:34.265794 3660 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 09:40:34.269356 10686 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:34.267607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.3.147.1:45141)
0812 09:40:34.268087 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:34.268093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:34.268115 (+ 22us) server_negotiation.cc:407] Connection header received
0812 09:40:34.268227 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:34.268232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:34.268419 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:34.268575 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:34.269173 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56130: BlockingRecv error: recv got EOF from 127.0.0.1:56130 (error 108)
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":116,"threads_started":1}
W20260812 09:40:34.269531 3660 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5cc' of type FILE
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:34.270674 3660 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:34.271593 3660 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:34.281083 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:34.272500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56140 (local address 127.3.147.1:45141)
0812 09:40:34.272686 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:34.272690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:34.272709 (+ 19us) server_negotiation.cc:407] Connection header received
0812 09:40:34.272829 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:34.272831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:34.272868 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:34.272933 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:34.273421 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.274157 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.274939 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.275363 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.275447 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:34.277779 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:34.277792 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:34.277810 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:34.277831 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:34.279295 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:34.279737 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:34.279750 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:34.279752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:34.279828 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:34.280113 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:34.280117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:34.280119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:34.280391 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:34.280599 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:34.280803 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 09:40:34.280956 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 09:40:34.284925 10686 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:34.283822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.3.147.1:45141)
0812 09:40:34.283949 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:34.283953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:34.284054 (+ 101us) server_negotiation.cc:407] Connection header received
0812 09:40:34.284237 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:34.284240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:34.284319 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:34.284406 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:34.284801 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56142: BlockingRecv error: recv got EOF from 127.0.0.1:56142 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 09:40:34.285091 3660 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5cc' of type FILE
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:34.286068 3660 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:34.287179 3660 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:34.295148 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:34.288022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.3.147.1:45141)
0812 09:40:34.288214 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:34.288218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:34.288232 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:34.288314 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:34.288316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:34.288346 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:34.288408 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:34.288852 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.289332 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.290085 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.290297 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.290372 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:34.292394 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:34.292397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:34.292399 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:34.292419 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:34.293817 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:34.294172 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:34.294174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:34.294175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:34.294242 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:34.294466 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:34.294468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:34.294469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:34.294607 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:34.294718 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:34.294857 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 09:40:34.294966 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 09:40:34.297828 10686 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:34.296937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56172 (local address 127.3.147.1:45141)
0812 09:40:34.297063 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:34.297066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:34.297146 (+ 80us) server_negotiation.cc:407] Connection header received
0812 09:40:34.297230 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:34.297232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:34.297264 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:34.297319 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:34.297732 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56172: BlockingRecv error: recv got EOF from 127.0.0.1:56172 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 09:40:34.297958 3660 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786527603645036-3660-0/krb5kdc/krb5cc' of type FILE
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:34.298906 3660 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:34.299790 3660 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:40:34.307710 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:34.300663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56174 (local address 127.3.147.1:45141)
0812 09:40:34.300814 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:34.300818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:34.300893 (+ 75us) server_negotiation.cc:407] Connection header received
0812 09:40:34.300972 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:34.300974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:34.301006 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:34.301067 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:34.301511 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.301982 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.302708 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:34.302906 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:34.302981 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:34.305046 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:34.305049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:34.305051 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:34.305080 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:34.306379 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:34.306750 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:34.306753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:34.306754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:34.306824 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:34.307047 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:34.307049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:34.307050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:34.307214 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:34.307329 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:34.307515 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 09:40:34.307603 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 09:40:34.308831 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 10212
I20260812 09:40:34.322732 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 10368
I20260812 09:40:34.330376 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 10522
I20260812 09:40:34.339289 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 10104
2026-08-12T09:40:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2356 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10700](info): set up 2 sockets
Aug 12 09:40:34 dist-test-slave-cbbz krb5kdc[10700](info): commencing operation
krb5kdc: starting...
W20260812 09:40:35.553073 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:35 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.62.keytab.
Entry for principal oryx/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.62.keytab.
I20260812 09:40:35.679600 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:34643
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.62.keytab
--principal=oryx/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:34643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:35.791739 10717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:35.791985 10717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:35.792026 10717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:35.795909 10717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:35.795955 10717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:35.795974 10717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:35.795990 10717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:35.796005 10717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:35.801024 10717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:34643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.62.keytab
--principal=oryx/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:34643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:35.801994 10717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:35.802942 10717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:35.811409 10723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:35.811385 10722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:35.811390 10726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:35.811429 10717 server_base.cc:1034] running on GCE node
I20260812 09:40:35.812340 10717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:35.813385 10717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:35.814543 10717 hybrid_clock.cc:648] HybridClock initialized: now 1786527635814525 us; error 31 us; skew 500 ppm
Aug 12 09:40:35 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527635, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:35.817425 10717 init.cc:377] Logged in from keytab as oryx/127.3.147.62@KRBTEST.COM (short username oryx)
I20260812 09:40:35.818266 10717 webserver.cc:460] Webserver started at http://127.3.147.62:44807/ using document root <none> and password file <none>
I20260812 09:40:35.818851 10717 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:35.818948 10717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:35.819195 10717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:35.821245 10717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "d890b142c12c469db8922f957d18eaf6"
format_stamp: "Formatted at 2026-08-12 09:40:35 on dist-test-slave-cbbz"
server_key: "825302641b93f5b4d541b07614c8e042"
server_key_iv: "2cab40e8ed98e11c220ac8ec78478c2f"
server_key_version: "encryptionkey@0"
I20260812 09:40:35.821836 10717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "d890b142c12c469db8922f957d18eaf6"
format_stamp: "Formatted at 2026-08-12 09:40:35 on dist-test-slave-cbbz"
server_key: "825302641b93f5b4d541b07614c8e042"
server_key_iv: "2cab40e8ed98e11c220ac8ec78478c2f"
server_key_version: "encryptionkey@0"
I20260812 09:40:35.825552 10717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:35.827905 10733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:35.829105 10717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:35.829250 10717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "d890b142c12c469db8922f957d18eaf6"
format_stamp: "Formatted at 2026-08-12 09:40:35 on dist-test-slave-cbbz"
server_key: "825302641b93f5b4d541b07614c8e042"
server_key_iv: "2cab40e8ed98e11c220ac8ec78478c2f"
server_key_version: "encryptionkey@0"
I20260812 09:40:35.829370 10717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:35.859601 10717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:35.863018 10717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:35.863237 10717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:35.870600 10717 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:34643
I20260812 09:40:35.870625 10796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:34643 every 8 connection(s)
I20260812 09:40:35.871532 10717 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:35.874373 10797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:35.876367 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 10717
I20260812 09:40:35.876511 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:35.876803 3660 external_mini_cluster.cc:1428] Setting key a879284e31b9df9eff6b9a5c3ee2ca68
I20260812 09:40:35.881664 10797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: Bootstrap starting.
I20260812 09:40:35.884465 10797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:35.885193 10797 log.cc:824] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:40:35 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:35.887351 10797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: No bootstrap required, opened a new log
I20260812 09:40:35.892223 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:35.878342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.3.147.62:34643)
0812 09:40:35.878793 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:35.878815 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:35.878859 (+ 44us) server_negotiation.cc:407] Connection header received
0812 09:40:35.879683 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:35.879709 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:35.880173 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:35.880548 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:35.881229 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:35.882687 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:35.883543 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:35.884262 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:35.884441 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:35.886955 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:35.886962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:35.886979 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:35.887015 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:35.889075 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:35.889517 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:35.889525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:35.889533 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:35.889704 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:35.889979 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:35.889982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:35.889984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:35.890489 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:35.890665 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:35.890913 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 09:40:35.891210 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":117,"threads_started":1}
I20260812 09:40:35.892184 10797 raft_consensus.cc:348] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } }
I20260812 09:40:35.892517 10797 raft_consensus.cc:374] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:35.892584 10797 raft_consensus.cc:729] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d890b142c12c469db8922f957d18eaf6, State: Initialized, Role: FOLLOWER
I20260812 09:40:35.893075 10797 consensus_queue.cc:260] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } }
I20260812 09:40:35.893204 10797 raft_consensus.cc:388] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:35.893328 10797 raft_consensus.cc:482] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:35.893453 10797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:35.894577 10797 raft_consensus.cc:504] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } }
I20260812 09:40:35.894914 10797 leader_election.cc:302] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d890b142c12c469db8922f957d18eaf6; no voters:
I20260812 09:40:35.895243 10797 leader_election.cc:288] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:35.895354 10803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:35.895627 10803 raft_consensus.cc:686] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 1 LEADER]: Becoming Leader. State: Replica: d890b142c12c469db8922f957d18eaf6, State: Running, Role: LEADER
I20260812 09:40:35.896000 10803 consensus_queue.cc:237] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } }
I20260812 09:40:35.896585 10797 sys_catalog.cc:564] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:35.897810 10805 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d890b142c12c469db8922f957d18eaf6. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } }
I20260812 09:40:35.897933 10805 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:35.898321 10815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:35.898861 10804 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } }
I20260812 09:40:35.898974 10804 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:35.901196 10815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:35.906795 10815 catalog_manager.cc:1347] Generated new cluster ID: a6d455f680644e5aa6d7e6f9bb4b121d
I20260812 09:40:35.906862 10815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:35.914433 10815 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:35.915608 10815 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:35.923322 10815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: Generated new TSK 0
I20260812 09:40:35.924072 10815 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab.
Entry for principal oryx/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab.
I20260812 09:40:35.959854 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34643
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab
--principal=oryx/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:36.083666 10830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:36.083954 10830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:36.084003 10830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:36.088244 10830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:36.088296 10830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:36.088353 10830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:36.093446 10830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab
--principal=oryx/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:36.094431 10830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:36.095400 10830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:36.105324 10840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:36.105314 10837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:36.105428 10830 server_base.cc:1034] running on GCE node
W20260812 09:40:36.105325 10838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:36.105967 10830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:36.106513 10830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:36.107705 10830 hybrid_clock.cc:648] HybridClock initialized: now 1786527636107669 us; error 56 us; skew 500 ppm
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:36.110569 10830 init.cc:377] Logged in from keytab as oryx/127.3.147.1@KRBTEST.COM (short username oryx)
I20260812 09:40:36.111348 10830 webserver.cc:460] Webserver started at http://127.3.147.1:40893/ using document root <none> and password file <none>
I20260812 09:40:36.111884 10830 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:36.111941 10830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:36.112102 10830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:36.114096 10830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "878000dad91247dab601fa220772870e"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "3ca16b7f44369269567f69724f7e9832"
server_key_iv: "5e35440c7ec01e5b720f2c8df9fbd9ca"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.114634 10830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "878000dad91247dab601fa220772870e"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "3ca16b7f44369269567f69724f7e9832"
server_key_iv: "5e35440c7ec01e5b720f2c8df9fbd9ca"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.118223 10830 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:40:36.120656 10848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.121904 10830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:36.122004 10830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "878000dad91247dab601fa220772870e"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "3ca16b7f44369269567f69724f7e9832"
server_key_iv: "5e35440c7ec01e5b720f2c8df9fbd9ca"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.122085 10830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:36.138906 10830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:36.142282 10830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:36.142446 10830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:36.142982 10830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:36.143831 10830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:36.143877 10830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.143950 10830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:36.143982 10830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.153244 10830 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:43785
I20260812 09:40:36.153267 10985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:43785 every 8 connection(s)
I20260812 09:40:36.154191 10830 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:36.158344 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 10830
I20260812 09:40:36.158478 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:36.158818 3660 external_mini_cluster.cc:1428] Setting key 168b41556e1cb8437c5543586554b218
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:40:36.169785 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.156051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:36225 (local address 127.3.147.62:34643)
0812 09:40:36.156220 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.156224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.157128 (+ 904us) server_negotiation.cc:407] Connection header received
0812 09:40:36.158353 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.158359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.158424 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.158529 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.159884 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.160598 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.162449 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.162724 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.162783 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.166004 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.166011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.166015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.166049 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.167642 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.168214 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.168219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.168220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.168303 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.168664 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.168667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.168669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.168849 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.168962 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.169461 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.169601 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 09:40:36.170603 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.156493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.1:36225)
0812 09:40:36.156954 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.156997 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.158138 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.158748 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.158755 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.159165 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.159736 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.159751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.160741 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.160747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.162283 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.162298 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.162948 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.162950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.163092 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.163768 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.163785 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.165776 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.167764 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.167770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.167784 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.168061 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.168408 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.168412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.168414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.168556 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.168962 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.168969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.169215 (+ 246us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.169402 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.169739 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":106,"threads_started":1}
I20260812 09:40:36.171928 10986 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34643
I20260812 09:40:36.172292 10986 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.2@KRBTEST.COM" created.
I20260812 09:40:36.172848 10986 heartbeater.cc:511] Master 127.3.147.62:34643 requested a full tablet report, sending...
I20260812 09:40:36.174399 10752 ts_manager.cc:194] Registered new tserver with Master: 878000dad91247dab601fa220772870e (127.3.147.1:43785)
I20260812 09:40:36.175588 10752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:36225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab.
Entry for principal oryx/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab.
I20260812 09:40:36.189111 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34643
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab
--principal=oryx/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:36.302781 10992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:36.303031 10992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:36.303107 10992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:36.307157 10992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:36.307209 10992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:36.307308 10992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:36.312469 10992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab
--principal=oryx/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:36.313449 10992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:36.314471 10992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:36.323825 11001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:36.323834 11004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:36.323845 11002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:36.323920 10992 server_base.cc:1034] running on GCE node
I20260812 09:40:36.324563 10992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:36.325173 10992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:36.326372 10992 hybrid_clock.cc:648] HybridClock initialized: now 1786527636326366 us; error 34 us; skew 500 ppm
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:36.329206 10992 init.cc:377] Logged in from keytab as oryx/127.3.147.2@KRBTEST.COM (short username oryx)
I20260812 09:40:36.330050 10992 webserver.cc:460] Webserver started at http://127.3.147.2:34675/ using document root <none> and password file <none>
I20260812 09:40:36.330621 10992 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:36.330732 10992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:36.330989 10992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:36.333205 10992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "37ddd4eba85240fbaa30d3cc8c3380fd"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "56c22a9f412df8eac6cec452076aed45"
server_key_iv: "1f2c0656bdda42e7d4b2a98ab6499dfd"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.333853 10992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "37ddd4eba85240fbaa30d3cc8c3380fd"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "56c22a9f412df8eac6cec452076aed45"
server_key_iv: "1f2c0656bdda42e7d4b2a98ab6499dfd"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.337636 10992 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:40:36.340193 11013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.341403 10992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:36.341547 10992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "37ddd4eba85240fbaa30d3cc8c3380fd"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "56c22a9f412df8eac6cec452076aed45"
server_key_iv: "1f2c0656bdda42e7d4b2a98ab6499dfd"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.341662 10992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:36.357816 10992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:36.360878 10992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:36.361089 10992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:36.361716 10992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:36.362635 10992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:36.362717 10992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.362818 10992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:36.362872 10992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.372660 10992 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:35333
I20260812 09:40:36.372706 11152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:35333 every 8 connection(s)
I20260812 09:40:36.373586 10992 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:36.376209 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 10992
I20260812 09:40:36.376371 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:36.376691 3660 external_mini_cluster.cc:1428] Setting key 7ce800b56b07d2c0ece4ee782d40c76f
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:40:36.388860 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.375389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:39779 (local address 127.3.147.62:34643)
0812 09:40:36.375545 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.375549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.376390 (+ 841us) server_negotiation.cc:407] Connection header received
0812 09:40:36.377790 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.377793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.377839 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.377940 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.379533 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.380049 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.381967 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.382192 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.382284 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.385173 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.385176 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.385178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.385201 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.386654 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.387200 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.387204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.387206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.387291 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.387693 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.387698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.387699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.387892 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.388021 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.388563 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.388683 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 09:40:36.389636 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.375721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.2:39779)
0812 09:40:36.376230 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.376294 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.377588 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.378146 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.378153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.378679 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.379372 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.379394 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.380328 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.380334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.381831 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.381846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.382362 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.382368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.382599 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.383211 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.383237 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.384987 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.386786 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.386792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.386805 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.387080 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.387386 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.387390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.387392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.387568 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.388020 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.388027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.388330 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.388520 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.388834 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":150,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.3@KRBTEST.COM" created.
I20260812 09:40:36.390859 11153 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34643
I20260812 09:40:36.391152 11153 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:36.391691 11153 heartbeater.cc:511] Master 127.3.147.62:34643 requested a full tablet report, sending...
I20260812 09:40:36.392876 10752 ts_manager.cc:194] Registered new tserver with Master: 37ddd4eba85240fbaa30d3cc8c3380fd (127.3.147.2:35333)
I20260812 09:40:36.393448 10752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:39779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab.
Entry for principal oryx/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab.
I20260812 09:40:36.406775 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34643
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab
--principal=oryx/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:36.520787 11159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:36.521075 11159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:36.521185 11159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:36.525454 11159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:36.525538 11159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:36.525624 11159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:36.531090 11159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab
--principal=oryx/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:34643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:36.532310 11159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:36.533452 11159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:36.543526 11166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:36.543521 11165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:36.543562 11169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:36.543970 11159 server_base.cc:1034] running on GCE node
I20260812 09:40:36.544466 11159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:36.545213 11159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:36.546435 11159 hybrid_clock.cc:648] HybridClock initialized: now 1786527636546421 us; error 24 us; skew 500 ppm
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:36.549397 11159 init.cc:377] Logged in from keytab as oryx/127.3.147.3@KRBTEST.COM (short username oryx)
I20260812 09:40:36.550343 11159 webserver.cc:460] Webserver started at http://127.3.147.3:39735/ using document root <none> and password file <none>
I20260812 09:40:36.550922 11159 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:36.550985 11159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:36.551229 11159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:36.553424 11159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "a4f38cb3e1864666adc0c5b202e8fd9a"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "6620a6699554e9f4e1df5b97cfaf5b1c"
server_key_iv: "0c400906d3b98beb954f7997dd36e1c2"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.554011 11159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "a4f38cb3e1864666adc0c5b202e8fd9a"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "6620a6699554e9f4e1df5b97cfaf5b1c"
server_key_iv: "0c400906d3b98beb954f7997dd36e1c2"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.558264 11159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:40:36.560932 11179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.562219 11159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:36.562366 11159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "a4f38cb3e1864666adc0c5b202e8fd9a"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "6620a6699554e9f4e1df5b97cfaf5b1c"
server_key_iv: "0c400906d3b98beb954f7997dd36e1c2"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.562490 11159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:36.583998 11159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:36.587059 11159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:36.587296 11159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:36.587946 11159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:36.588919 11159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:36.588996 11159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.589083 11159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:36.589140 11159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.598953 11159 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42599
I20260812 09:40:36.598980 11314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42599 every 8 connection(s)
I20260812 09:40:36.599851 11159 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:36.603571 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 11159
I20260812 09:40:36.603730 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:36.604041 3660 external_mini_cluster.cc:1428] Setting key 4c0a8c43bf7ec3decbf571bde5857136
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:36.613883 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.601730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:56097 (local address 127.3.147.62:34643)
0812 09:40:36.601900 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.601904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.602675 (+ 771us) server_negotiation.cc:407] Connection header received
0812 09:40:36.604113 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.604117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.604200 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.604320 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.605585 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.606213 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.607347 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.607586 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.607709 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.610144 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.610148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.610150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.610172 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.611739 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.612314 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.612318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.612320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.612403 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.612756 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.612761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.612762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.612940 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.613050 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.613535 (+ 485us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.613649 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 09:40:36.614670 11317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.602061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.3:56097)
0812 09:40:36.602522 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.602563 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.603901 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.604495 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.604502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.604969 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.605440 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.605453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.606339 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.606342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.607217 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.607226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.607750 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.607755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.607932 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.608508 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.608533 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.609998 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.611851 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.611856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.611867 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.612118 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.612510 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.612513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.612515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.612652 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.613064 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.613072 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.613300 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.613490 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.613816 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":104,"threads_started":1}
I20260812 09:40:36.615792 11315 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34643
I20260812 09:40:36.616047 11315 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:36.616613 11315 heartbeater.cc:511] Master 127.3.147.62:34643 requested a full tablet report, sending...
I20260812 09:40:36.617884 10752 ts_manager.cc:194] Registered new tserver with Master: a4f38cb3e1864666adc0c5b202e8fd9a (127.3.147.3:42599)
I20260812 09:40:36.618444 10752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:56097
I20260812 09:40:36.629696 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.147.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.61.keytab.
Entry for principal oryx/127.3.147.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.61.keytab.
I20260812 09:40:36.664110 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.61:33971
--webserver_interface=127.3.147.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.61.keytab
--principal=oryx/127.3.147.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:34643,127.3.147.61:33971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:36.779358 11321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:36.779619 11321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:36.779690 11321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:36.783685 11321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:36.783741 11321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:36.783761 11321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:36.783776 11321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:36.783823 11321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:36.788988 11321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:34643,127.3.147.61:33971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.61.keytab
--principal=oryx/127.3.147.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.61:33971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.147.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
W20260812 09:40:36.789202 11321 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.147.62:34643,127.3.147.61:33971'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 09:40:36.790052 11321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:36.791092 11321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:36.800122 11331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:36.800290 11326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:36.800525 11321 server_base.cc:1034] running on GCE node
W20260812 09:40:36.800125 11328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:36.801138 11321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:36.802270 11321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:36.803478 11321 hybrid_clock.cc:648] HybridClock initialized: now 1786527636803438 us; error 54 us; skew 500 ppm
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:36.806450 11321 init.cc:377] Logged in from keytab as oryx/127.3.147.61@KRBTEST.COM (short username oryx)
I20260812 09:40:36.807279 11321 webserver.cc:460] Webserver started at http://127.3.147.61:36075/ using document root <none> and password file <none>
I20260812 09:40:36.807791 11321 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:36.807850 11321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:36.808014 11321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:36.810065 11321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data/instance:
uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "90cf960d8feb4d44f56632e177b26b21"
server_key_iv: "746f8d06469cf724e3077819252392b9"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.810653 11321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal/instance:
uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "90cf960d8feb4d44f56632e177b26b21"
server_key_iv: "746f8d06469cf724e3077819252392b9"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.814392 11321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 09:40:36.816747 11343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.817903 11321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:40:36.818005 11321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "90cf960d8feb4d44f56632e177b26b21"
server_key_iv: "746f8d06469cf724e3077819252392b9"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.818084 11321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:36.850687 11321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:36.854178 11321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:36.854336 11321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:36.861312 11321 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.61:33971
I20260812 09:40:36.861613 11403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.61:33971 every 8 connection(s)
I20260812 09:40:36.862337 11321 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
I20260812 09:40:36.865195 11404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:36.868983 11404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:40:36.871517 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 11321
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.61@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.147.61@KRBTEST.COM
I20260812 09:40:36.884204 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.870870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.3.147.62:34643)
0812 09:40:36.871049 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.871055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.871960 (+ 905us) server_negotiation.cc:407] Connection header received
0812 09:40:36.873202 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.873205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.873266 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.873343 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.874297 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.875143 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.876608 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.876923 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.877036 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.880436 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.880438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.880440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.880462 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.882053 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.882677 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.882680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.882682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.882765 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.883122 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.883130 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.883132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.883326 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.883439 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.883834 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.883963 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 09:40:36.885033 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.872447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.3.147.61:33971)
0812 09:40:36.872809 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.872821 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.872861 (+ 40us) server_negotiation.cc:407] Connection header received
0812 09:40:36.873009 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.873027 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.873279 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.873462 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.874343 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.875560 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.876490 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.877075 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.877267 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.880365 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.880371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.880383 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.880407 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.882224 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.882700 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.882705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.882706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.882827 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.883121 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.883125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.883126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.883596 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.883780 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.884543 (+ 763us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.884711 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":277,"thread_start_us":82,"threads_started":1}
I20260812 09:40:36.885040 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.871321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:48546)
0812 09:40:36.871808 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.871860 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.872984 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.873413 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.873418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.873861 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.874142 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.874155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.875284 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.875287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.876484 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.876495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.877077 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.877081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.877269 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.878050 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.878069 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.880247 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.882190 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.882202 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.882212 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.882545 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.882865 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.882871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.882875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.883007 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.883438 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.883444 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.883611 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.883776 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.884098 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":42,"spinlock_wait_cycles":36096,"thread_start_us":137,"threads_started":1}
I20260812 09:40:36.886562 11404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527636, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.61@KRBTEST.COM for oryx/127.3.147.61@KRBTEST.COM
I20260812 09:40:36.894170 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.887086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.0.0.1:60788)
0812 09:40:36.887247 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.887262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.887336 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.887587 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.887590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.887748 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.887827 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.887832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.888619 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.888622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.889097 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.889104 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.889555 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.889559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.889688 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.890136 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.890148 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.891308 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.893022 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.893024 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.893025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.893221 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.893453 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.893454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.893456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.893523 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.893823 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.893825 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.893890 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.893964 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.894046 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":3}
I20260812 09:40:36.894220 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.887175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.3.147.61:33971)
0812 09:40:36.887401 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.887407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.887421 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:36.887456 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.887458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.887487 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.887573 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.887938 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.888503 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.889213 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.889449 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.889559 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.891460 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.891463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.891465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.891485 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.892923 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.893303 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.893306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.893307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.893375 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.893604 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.893606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.893607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.893745 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.893816 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.894019 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.894106 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":148}
I20260812 09:40:36.897650 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Bootstrap starting.
I20260812 09:40:36.899994 11404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:36.900779 11404 log.cc:824] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:36.902719 11404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: No bootstrap required, opened a new log
I20260812 09:40:36.905936 11404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } }
I20260812 09:40:36.906179 11404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:36.906286 11404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77bfd6eb3dbc4abaaadcf925c64e2bca, State: Initialized, Role: FOLLOWER
I20260812 09:40:36.906729 11404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } }
I20260812 09:40:36.907246 11410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } } }
I20260812 09:40:36.907351 11410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:40:36.907935 11404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: configured and running, proceeding with master startup.
W20260812 09:40:36.912830 11423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:40:36.912921 11423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:40:36.915436 11321 master_runner.cc:428] Detected that this master 77bfd6eb3dbc4abaaadcf925c64e2bca is joining an existing cluster
I20260812 09:40:36.915501 11321 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.147.61:33971
I20260812 09:40:36.926630 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.918266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.3.147.61:33971)
0812 09:40:36.918388 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.918393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.918408 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:40:36.918445 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.918450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.918501 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.918576 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.919351 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.920192 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.921538 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.921905 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.922000 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.923268 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.923271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.923274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.923291 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.924952 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.925484 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.925487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.925489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.925552 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.925788 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.925790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.925792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.925965 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.926021 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.926366 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.926468 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:40:36.927268 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.917863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.0.0.1:60802)
0812 09:40:36.918229 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.918244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.918334 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.918691 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.918693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.918995 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.919211 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.919230 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.920326 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.920329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.921383 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.921393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.922113 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.922116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.922198 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.922715 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.922728 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.923147 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.925116 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.925119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.925121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.925392 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.925647 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.925649 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.925652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.925717 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.926070 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.926073 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.926184 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.926987 (+ 803us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.927104 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":44,"spinlock_wait_cycles":2304,"thread_start_us":92,"threads_started":1}
I20260812 09:40:36.929756 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.917863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:48562)
0812 09:40:36.918429 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.918442 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.918530 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.918776 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.918778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.918995 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.919218 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.919228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.922248 (+ 3020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.922252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.923768 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.923779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.924889 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.924892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.924999 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.925590 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.925609 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.926025 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.928127 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.928130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.928132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.928389 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.928716 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.928718 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.928721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.928799 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.929227 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.929230 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.929337 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.929455 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.929588 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":52,"spinlock_wait_cycles":1280,"thread_start_us":77,"threads_started":1}
I20260812 09:40:36.929778 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.917935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.3.147.62:34643)
0812 09:40:36.918057 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.918062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.918529 (+ 467us) server_negotiation.cc:407] Connection header received
0812 09:40:36.918608 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.918612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.918658 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.918743 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.920331 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.922114 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.923925 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.924540 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.924715 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.926185 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.926189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.926192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.926235 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.927989 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.928506 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.928510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.928512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.928604 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.928905 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.928909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.928911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.929115 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.929210 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.929516 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.929630 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:40:36.935662 11410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:36.935834 11410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } }
I20260812 09:40:36.936717 11410 leader_election.cc:288] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d890b142c12c469db8922f957d18eaf6 (127.3.147.62:34643)
I20260812 09:40:36.937162 10764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d890b142c12c469db8922f957d18eaf6" is_pre_election: true
I20260812 09:40:36.937810 11346 leader_election.cc:302] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 77bfd6eb3dbc4abaaadcf925c64e2bca; no voters: d890b142c12c469db8922f957d18eaf6
I20260812 09:40:36.938092 11410 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 12 09:40:36 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527635, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.62@KRBTEST.COM for oryx/127.3.147.61@KRBTEST.COM
I20260812 09:40:36.944233 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.934566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60814 (local address 127.3.147.61:33971)
0812 09:40:36.934690 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.934695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.935013 (+ 318us) server_negotiation.cc:407] Connection header received
0812 09:40:36.935205 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.935208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.935258 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.935342 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.936180 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.936663 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.937922 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.938252 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.938353 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.940669 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.940673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.940675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.940699 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.942493 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.943038 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.943041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.943042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.943112 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.943404 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.943406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.943407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.943554 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.943628 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.943984 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.944066 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:40:36.944306 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.934514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.0.0.1:60814)
0812 09:40:36.934883 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.934933 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.935073 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.935381 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.935385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.935857 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.936013 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.936022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.936886 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.936889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.937793 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.937807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.938376 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.938379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.938488 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.939044 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.939063 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.940530 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.942625 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.942635 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.942641 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.942932 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.943213 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.943217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.943219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.943313 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.943661 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.943668 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.943789 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.943937 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.944123 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":113,"threads_started":1}
I20260812 09:40:36.945748 10752 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.147.61:33971: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca"
member_type: NON_VOTER
last_known_addr {
host: "127.3.147.61"
port: 33971
}
attrs {
promote: true
}
}
dest_uuid: "d890b142c12c469db8922f957d18eaf6"
cas_config_opid_index: -1
I20260812 09:40:36.946434 10752 consensus_queue.cc:237] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } }
I20260812 09:40:36.948333 11437 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 77bfd6eb3dbc4abaaadcf925c64e2bca (127.3.147.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } }
I20260812 09:40:36.948958 11371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:36.949869 11437 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.147.61:33971
I20260812 09:40:36.950201 11437 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } }
I20260812 09:40:36.950304 11437 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:36.950327 11321 master.cc:537] Master@127.3.147.61:33971 shutting down...
WARNING: no policy specified for oryx/127.3.147.60@KRBTEST.COM; defaulting to no policy
I20260812 09:40:36.950519 11371 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Refusing update from remote peer d890b142c12c469db8922f957d18eaf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 09:40:36.951164 11434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 09:40:36.954490 10735 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 09:40:36.955209 11321 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 09:40:36.955370 10735 consensus_peers.cc:487] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 -> Peer 77bfd6eb3dbc4abaaadcf925c64e2bca (127.3.147.61:33971): Couldn't send request to peer 77bfd6eb3dbc4abaaadcf925c64e2bca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 09:40:36.955502 11321 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:36.955606 11321 tablet_replica.cc:330] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.60@KRBTEST.COM" created.
I20260812 09:40:36.970386 11321 master.cc:552] Master@127.3.147.61:33971 shutdown complete.
I20260812 09:40:36.970443 11321 master_runner.cc:305] Clearing existing system tablet
I20260812 09:40:36.970706 11321 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:36.970762 11321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:36.972103 11321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 09:40:36.973172 11442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:36.973403 11321 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:36.973552 11321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:36.973611 11321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "90cf960d8feb4d44f56632e177b26b21"
server_key_iv: "746f8d06469cf724e3077819252392b9"
server_key_version: "encryptionkey@0"
I20260812 09:40:36.976042 11321 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:36.978826 11321 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 09:40:36.978905 11321 log.cc:1196] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 09:40:36.979256 11321 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Deleting consensus metadata
I20260812 09:40:36.979450 11321 master_runner.cc:315] Copying system tablet from 127.3.147.62:34643
I20260812 09:40:36.980891 11321 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: tablet copy: Beginning tablet copy session from remote peer at address 127.3.147.62:34643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.60.keytab.
Entry for principal oryx/127.3.147.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.60.keytab.
I20260812 09:40:36.984807 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.60:41979
--webserver_interface=127.3.147.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.60.keytab
--principal=oryx/127.3.147.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:34643,127.3.147.61:33971,127.3.147.60:41979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:40:36.990861 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.981984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.3.147.62:34643)
0812 09:40:36.982146 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:36.982151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:36.982164 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:36.982212 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:36.982214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:36.982252 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:36.982374 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:36.983379 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.984317 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.985564 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.986003 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.986123 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.987506 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:36.987510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:36.987513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:36.987545 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:36.989138 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.989589 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.989596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.989600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.989684 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:36.989976 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:36.989981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:36.989983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:36.990179 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:36.990298 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:36.990569 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 09:40:36.990680 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 09:40:36.990875 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:36.981599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:48574)
0812 09:40:36.981894 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:36.981910 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:36.982037 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:36.982393 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:36.982395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:36.982607 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:36.983151 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.983166 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.984458 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.984461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.985410 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:36.985426 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:36.986147 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:36.986153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:36.986285 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:36.986942 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:36.986954 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:36.987325 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:36.989266 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.989270 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.989272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.989483 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.989787 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:36.989790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:36.989792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:36.989879 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:36.990288 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:36.990291 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:36.990400 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 09:40:36.990514 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 09:40:36.990682 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260812 09:40:36.991452 10776 tablet_copy_service.cc:140] P d890b142c12c469db8922f957d18eaf6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 77bfd6eb3dbc4abaaadcf925c64e2bca ({username='oryx', principal='oryx/127.3.147.61@KRBTEST.COM'} at 127.0.0.1:48574)
I20260812 09:40:36.991619 10776 tablet_copy_service.cc:161] P d890b142c12c469db8922f957d18eaf6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 77bfd6eb3dbc4abaaadcf925c64e2bca at {username='oryx', principal='oryx/127.3.147.61@KRBTEST.COM'} at 127.0.0.1:48574: session id = 77bfd6eb3dbc4abaaadcf925c64e2bca-00000000000000000000000000000000
I20260812 09:40:36.993310 10776 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 09:40:36.995010 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 09:40:36.997428 11321 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: tablet copy: Starting download of 0 data blocks...
I20260812 09:40:36.997646 11321 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: tablet copy: Starting download of 1 WAL segments...
I20260812 09:40:36.999411 11321 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 09:40:37.001174 10776 tablet_copy_service.cc:342] P d890b142c12c469db8922f957d18eaf6: Request end of tablet copy session 77bfd6eb3dbc4abaaadcf925c64e2bca-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.147.61@KRBTEST.COM'} at 127.0.0.1:48574
I20260812 09:40:37.001281 10776 tablet_copy_service.cc:433] P d890b142c12c469db8922f957d18eaf6: ending tablet copy session 77bfd6eb3dbc4abaaadcf925c64e2bca-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 77bfd6eb3dbc4abaaadcf925c64e2bca
I20260812 09:40:37.008682 11321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:37.010454 11453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:37.010496 11450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:37.010818 11451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:37.012307 11321 server_base.cc:1034] running on GCE node
I20260812 09:40:37.012486 11321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:37.012810 11321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:37.013954 11321 hybrid_clock.cc:648] HybridClock initialized: now 1786527637013963 us; error 44 us; skew 500 ppm
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:37.015514 11321 init.cc:377] Logged in from keytab as oryx/127.3.147.61@KRBTEST.COM (short username oryx)
I20260812 09:40:37.015980 11321 webserver.cc:460] Webserver started at http://127.3.147.61:42657/ using document root <none> and password file <none>
I20260812 09:40:37.016207 11321 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:37.016258 11321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:37.017542 11321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:37.018477 11462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:37.018718 11321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:37.018817 11321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "90cf960d8feb4d44f56632e177b26b21"
server_key_iv: "746f8d06469cf724e3077819252392b9"
server_key_version: "encryptionkey@0"
I20260812 09:40:37.018921 11321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:37.027510 11321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:37.029615 11321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:37.035173 11321 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.61:33971
I20260812 09:40:37.035667 11321 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
I20260812 09:40:37.036393 11523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.61:33971 every 8 connection(s)
I20260812 09:40:37.037431 11524 sys_catalog.cc:263] Verifying existing consensus state
I20260812 09:40:37.038246 11524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Bootstrap starting.
I20260812 09:40:37.047132 11524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:40:37.047451 11524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Bootstrap complete.
I20260812 09:40:37.048115 11524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } }
I20260812 09:40:37.048309 11524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 77bfd6eb3dbc4abaaadcf925c64e2bca, State: Initialized, Role: LEARNER
I20260812 09:40:37.048434 11524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } }
I20260812 09:40:37.048803 11527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } }
I20260812 09:40:37.048995 11524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:37.049003 11527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: This master's current role is: LEARNER
I20260812 09:40:37.052126 11540 catalog_manager.cc:1259] Loaded cluster ID: a6d455f680644e5aa6d7e6f9bb4b121d
I20260812 09:40:37.052266 11540 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: loading cluster ID for follower catalog manager: success
I20260812 09:40:37.053959 11540 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: acquiring CA information for follower catalog manager: success
I20260812 09:40:37.055104 11540 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 09:40:37.150377 11449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:37.150900 11449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:37.151041 11449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:37.157482 11449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:37.157610 11449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:37.157704 11449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:37.157814 11449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:37.157902 11449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:37.166569 11449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:34643,127.3.147.61:33971,127.3.147.60:41979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.60.keytab
--principal=oryx/127.3.147.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.60:41979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.147.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:37.168442 11449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:37.170204 11449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:40:37.178210 10986 heartbeater.cc:503] Master 127.3.147.62:34643 was elected leader, sending a full tablet report...
W20260812 09:40:37.187114 11551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:37.187206 11548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:37.187250 11549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:37.188403 11449 server_base.cc:1034] running on GCE node
I20260812 09:40:37.189357 11449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:37.191277 11449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:37.192584 11449 hybrid_clock.cc:648] HybridClock initialized: now 1786527637192544 us; error 51 us; skew 500 ppm
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:37.198781 11449 init.cc:377] Logged in from keytab as oryx/127.3.147.60@KRBTEST.COM (short username oryx)
I20260812 09:40:37.200444 11449 webserver.cc:460] Webserver started at http://127.3.147.60:33347/ using document root <none> and password file <none>
I20260812 09:40:37.201478 11449 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:37.201645 11449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:37.202118 11449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:37.206363 11449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data/instance:
uuid: "14c4890dce4b4b68aebc5fefe3034a61"
format_stamp: "Formatted at 2026-08-12 09:40:37 on dist-test-slave-cbbz"
server_key: "3fede691b0a3263f58ec2131320f953d"
server_key_iv: "2aa71e695b47b6cc45a34b8725bcd486"
server_key_version: "encryptionkey@0"
I20260812 09:40:37.207470 11449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal/instance:
uuid: "14c4890dce4b4b68aebc5fefe3034a61"
format_stamp: "Formatted at 2026-08-12 09:40:37 on dist-test-slave-cbbz"
server_key: "3fede691b0a3263f58ec2131320f953d"
server_key_iv: "2aa71e695b47b6cc45a34b8725bcd486"
server_key_version: "encryptionkey@0"
I20260812 09:40:37.214361 11449 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260812 09:40:37.218457 11561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:37.220796 11449 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 09:40:37.221024 11449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
uuid: "14c4890dce4b4b68aebc5fefe3034a61"
format_stamp: "Formatted at 2026-08-12 09:40:37 on dist-test-slave-cbbz"
server_key: "3fede691b0a3263f58ec2131320f953d"
server_key_iv: "2aa71e695b47b6cc45a34b8725bcd486"
server_key_version: "encryptionkey@0"
I20260812 09:40:37.221202 11449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:37.248420 11449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:37.254472 11449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:37.254933 11449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:37.283115 11449 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.60:41979
I20260812 09:40:37.283771 11621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.60:41979 every 8 connection(s)
I20260812 09:40:37.284794 11449 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
I20260812 09:40:37.287515 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 11449
I20260812 09:40:37.291433 11622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:37.302882 11622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.147.60@KRBTEST.COM
I20260812 09:40:37.315053 11624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.290338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.3.147.60:41979)
0812 09:40:37.291368 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.291436 (+ 68us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.291496 (+ 60us) server_negotiation.cc:407] Connection header received
0812 09:40:37.293225 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.293249 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.293807 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.294303 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.296602 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.299501 (+ 2899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.301030 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.302302 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.302601 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.306960 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.306966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.306992 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.307042 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.310639 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.311403 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.311414 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.311419 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.311587 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.312066 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.312068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.312069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.312758 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.313199 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.313460 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.313858 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":400,"threads_started":1}
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.60@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:37.322605 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.306493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.3.147.62:34643)
0812 09:40:37.306740 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.306748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.307534 (+ 786us) server_negotiation.cc:407] Connection header received
0812 09:40:37.307754 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.307756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.307870 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.308042 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.309805 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.310922 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.312770 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.313304 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.313460 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.317777 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.317781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.317784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.317842 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.320235 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.320846 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.320849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.320850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.320945 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.321347 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.321350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.321351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.321574 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.321680 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.322321 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.322448 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 09:40:37.322882 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.306218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:48588)
0812 09:40:37.307302 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.307394 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.307598 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.308096 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.308113 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.309212 (+ 1099us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:37.309607 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.309615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.311085 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.311092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.312618 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.312634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.313487 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.313493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.313963 (+ 470us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.314965 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:37.314986 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:37.317561 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:37.320381 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.320394 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.320399 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.320727 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.321102 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.321110 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.321112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.321231 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.321729 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:37.321742 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:37.321977 (+ 235us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.322245 (+ 268us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.322470 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":418,"threads_started":1}
I20260812 09:40:37.324429 11622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.60@KRBTEST.COM for oryx/127.3.147.61@KRBTEST.COM
I20260812 09:40:37.337366 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.325162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.0.0.1:60820)
0812 09:40:37.325331 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.325361 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.325455 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.326058 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.326067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.326392 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:37.326713 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.326721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.328033 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.328039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.329045 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.329060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.329839 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.329844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.330067 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.330579 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:37.330609 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:37.332431 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:37.334971 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.334984 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.334992 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.335288 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.335733 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.335739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.335750 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.335865 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.336549 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:37.336553 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:37.336701 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.336894 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.337070 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 09:40:37.337414 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.325268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.3.147.61:33971)
0812 09:40:37.325732 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.325735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.325753 (+ 18us) server_negotiation.cc:407] Connection header received
0812 09:40:37.325805 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.325807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.325878 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.326023 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.326892 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.327814 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.329258 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.329653 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.329853 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.332651 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.332659 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.332661 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.332696 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.334760 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.335454 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.335460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.335462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.335582 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.336017 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.336026 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.336028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.336390 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.336571 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.336984 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.337127 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":210,"threads_started":1}
I20260812 09:40:37.338653 11622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.60@KRBTEST.COM for oryx/127.3.147.60@KRBTEST.COM
I20260812 09:40:37.350658 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.339351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.0.0.1:34104)
0812 09:40:37.339485 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.339503 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.339580 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.339916 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.339918 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.340108 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:37.340264 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.340276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.341981 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.341983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.342659 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.342673 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.343330 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.343334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.343436 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.343968 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:37.343988 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:37.345739 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:37.348375 (+ 2636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.348379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.348381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.348746 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.349160 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.349166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.349169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.349293 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.349889 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:37.349892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:37.350005 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.350190 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.350391 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":1}
I20260812 09:40:37.350679 11624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.339506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.3.147.60:41979)
0812 09:40:37.339672 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.339675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.339692 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:40:37.339734 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.339737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.339793 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.339882 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.340393 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.341769 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.342787 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.343193 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.343326 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.346012 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.346016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.346018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.346052 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.348090 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.348909 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.348917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.348921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.349018 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.349441 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.349455 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.349459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.349700 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.349889 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.350272 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.350497 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":68}
I20260812 09:40:37.355854 11622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Bootstrap starting.
I20260812 09:40:37.360450 11622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:37.362183 11622 log.cc:824] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:37.365445 11622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: No bootstrap required, opened a new log
I20260812 09:40:37.373625 11622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } }
I20260812 09:40:37.374090 11622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:37.374195 11622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c4890dce4b4b68aebc5fefe3034a61, State: Initialized, Role: FOLLOWER
I20260812 09:40:37.375028 11622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } }
I20260812 09:40:37.376238 11628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } } }
I20260812 09:40:37.376549 11628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:40:37.377679 11622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 09:40:37.389488 11643 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:40:37.389621 11643 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:40:37.394560 11449 master_runner.cc:428] Detected that this master 14c4890dce4b4b68aebc5fefe3034a61 is joining an existing cluster
I20260812 09:40:37.394663 11449 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.147.60:41979
I20260812 09:40:37.395462 11153 heartbeater.cc:503] Master 127.3.147.62:34643 was elected leader, sending a full tablet report...
I20260812 09:40:37.427171 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 10830
I20260812 09:40:37.451436 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.399250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.3.147.62:34643)
0812 09:40:37.399495 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.399508 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.400895 (+ 1387us) server_negotiation.cc:407] Connection header received
0812 09:40:37.401171 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.401174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.401234 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.401312 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.403111 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.404123 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.408127 (+ 4004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.408650 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.408793 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.442079 (+ 33286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.442090 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.442108 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.442203 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.447404 (+ 5201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.448391 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.448406 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.448408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.448541 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.449169 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.449173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.449175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.449853 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.449974 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.451069 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.451260 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 09:40:37.451519 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:43785
--local_ip_for_outbound_sockets=127.3.147.1
--tserver_master_addrs=127.3.147.62:34643
--webserver_port=40893
--webserver_interface=127.3.147.1
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab
--principal=oryx/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971,127.3.147.60:41979 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:40:37.452497 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.399571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:48592)
0812 09:40:37.400771 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.400796 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.400984 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.401503 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.401507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.402225 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:37.402903 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.402922 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.404379 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.404385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.407889 (+ 3504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.407907 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.409146 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.409156 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.409490 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.410553 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:37.410603 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:37.441574 (+ 30971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:37.447748 (+ 6174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.447751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.447754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.448201 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.448873 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.448876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.448878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.449017 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.450296 (+ 1279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:37.450301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:37.450752 (+ 451us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.451769 (+ 1017us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.452112 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":572,"spinlock_wait_cycles":2944,"thread_start_us":739,"threads_started":1}
I20260812 09:40:37.462586 11654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.401518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.0.0.1:34114)
0812 09:40:37.402420 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.402442 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.402638 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.403141 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.403145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.403743 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:37.404271 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.404290 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.406087 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.406097 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.407912 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.407929 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.409100 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.409105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.409573 (+ 468us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.411302 (+ 1729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:37.411334 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:37.442920 (+ 31586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:37.458195 (+ 15275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.458210 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.458212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.458837 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.459553 (+ 716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.459560 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.459565 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.459875 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.461059 (+ 1184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:37.461064 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:37.461352 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.461602 (+ 250us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.461969 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":560,"thread_start_us":420,"threads_started":1}
I20260812 09:40:37.463639 11624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.401009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34114 (local address 127.3.147.60:41979)
0812 09:40:37.401170 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.401175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.402554 (+ 1379us) server_negotiation.cc:407] Connection header received
0812 09:40:37.402770 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.402774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.402836 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.402926 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.404482 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.405642 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.408161 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.408663 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.408795 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.443318 (+ 34523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.443324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.443327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.443375 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.458000 (+ 14625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.459046 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.459052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.459054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.459308 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.460242 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.460246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.460248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.460733 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.460987 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.463056 (+ 2069us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.463261 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 09:40:37.466563 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.399314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.3.147.61:33971)
0812 09:40:37.399512 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.399521 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.400701 (+ 1180us) server_negotiation.cc:407] Connection header received
0812 09:40:37.400994 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.400998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.401093 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.401188 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.403099 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.404123 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.408127 (+ 4004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.408674 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.408797 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.443323 (+ 34526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.443343 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.443349 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.443424 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.455382 (+ 11958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.459647 (+ 4265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.459664 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.459667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.459936 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.464447 (+ 4511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.464452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.464454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.464942 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.465127 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.465900 (+ 773us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.466238 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 09:40:37.467988 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.399528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.0.0.1:60836)
0812 09:40:37.400509 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.400538 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.400806 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.401403 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.401409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.402298 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:37.402839 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.402870 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.404379 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.404385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.407889 (+ 3504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.407907 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.409144 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.409156 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.410686 (+ 1530us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.441479 (+ 30793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:37.441539 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:37.442807 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:37.456253 (+ 13446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.456257 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.456260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.458889 (+ 2629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.462655 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.462663 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.463341 (+ 678us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.463592 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.465241 (+ 1649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:37.465244 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:37.465414 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.466889 (+ 1475us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.467225 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":436,"spinlock_wait_cycles":1792,"thread_start_us":316,"threads_started":1}
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527635, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.62@KRBTEST.COM for oryx/127.3.147.60@KRBTEST.COM
I20260812 09:40:37.482913 11624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.467340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34130 (local address 127.3.147.60:41979)
0812 09:40:37.467568 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.467589 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.468770 (+ 1181us) server_negotiation.cc:407] Connection header received
0812 09:40:37.469112 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.469117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.469183 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.469379 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.470324 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.471086 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.472373 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.472680 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.472877 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.476443 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.476455 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.476466 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.476500 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.479477 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.480336 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.480340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.480342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.480474 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.481031 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.481035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.481036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.481410 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.481701 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.482345 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.482555 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 09:40:37.482935 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.466665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.0.0.1:34130)
0812 09:40:37.468653 (+ 1988us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.468681 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.468896 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.469404 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.469407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.469988 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:37.470138 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.470153 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.471269 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.471276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.472081 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.472098 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.472878 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.472882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.473020 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.473871 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:37.473896 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:37.476104 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:37.479760 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.479770 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.479783 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.480104 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.480665 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.480668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.480670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.480831 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.481675 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:37.481679 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:37.481914 (+ 235us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.482143 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.482598 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1325,"thread_start_us":146,"threads_started":1}
I20260812 09:40:37.485188 10752 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.147.60:41979: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61"
member_type: NON_VOTER
last_known_addr {
host: "127.3.147.60"
port: 41979
}
attrs {
promote: true
}
}
dest_uuid: "d890b142c12c469db8922f957d18eaf6"
cas_config_opid_index: 5
I20260812 09:40:37.486258 10752 consensus_queue.cc:237] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } }
I20260812 09:40:37.489195 11659 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.490656 11593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:37.492854 11659 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.147.60:41979
I20260812 09:40:37.493621 11449 master.cc:537] Master@127.3.147.60:41979 shutting down...
I20260812 09:40:37.493691 11593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 FOLLOWER]: Refusing update from remote peer d890b142c12c469db8922f957d18eaf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 09:40:37.502106 11659 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:37.506321 11658 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.506578 11658 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:37.507750 11449 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:37.508405 11449 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:37.508698 11449 tablet_replica.cc:330] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: stopping tablet replica
I20260812 09:40:37.509636 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.488674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.0.0.1:60842)
0812 09:40:37.495481 (+ 6807us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.495512 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.495671 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.496733 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.496747 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.496792 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:37.497219 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.497250 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.499650 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.499655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.504513 (+ 4858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.504535 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.508403 (+ 3868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.508409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.508849 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.508945 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.509102 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.509397 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6653}
I20260812 09:40:37.510596 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.494601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60842 (local address 127.3.147.61:33971)
0812 09:40:37.495028 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.495063 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.495961 (+ 898us) server_negotiation.cc:407] Connection header received
0812 09:40:37.496092 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.496096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.496217 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.496351 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:37.497768 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.499474 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.504871 (+ 5397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.507105 (+ 2234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.507253 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.507558 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.509857 (+ 2299us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.510098 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260812 09:40:37.511523 11488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 LEARNER]: Refusing update from remote peer d890b142c12c469db8922f957d18eaf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 09:40:37.512481 11657 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 09:40:37.516104 11488 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
W20260812 09:40:37.517616 10736 consensus_peers.cc:487] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 -> Peer 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60:41979): Couldn't send request to peer 14c4890dce4b4b68aebc5fefe3034a61. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 09:40:37.523408 11666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.523715 11666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: This master's current role is: LEARNER
I20260812 09:40:37.523638 11527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: SysCatalogTable state changed. Reason: New leader d890b142c12c469db8922f957d18eaf6. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: NON_VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: true } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.523834 11527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: This master's current role is: LEARNER
I20260812 09:40:37.524732 11661 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: attempting to promote NON_VOTER 77bfd6eb3dbc4abaaadcf925c64e2bca to VOTER
I20260812 09:40:37.525498 11661 consensus_queue.cc:237] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } }
I20260812 09:40:37.528512 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.527896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.0.0.1:34138)
0812 09:40:37.528208 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.528357 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 09:40:37.528496 11488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 LEARNER]: Refusing update from remote peer d890b142c12c469db8922f957d18eaf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 09:40:37.529976 11657 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 09:40:37.531266 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.530803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.0.0.1:34146)
0812 09:40:37.531070 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.531156 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260812 09:40:37.531677 10736 consensus_peers.cc:487] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 -> Peer 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60:41979): Couldn't send request to peer 14c4890dce4b4b68aebc5fefe3034a61. Status: Network error: Client connection negotiation failed: client connection to 127.3.147.60:41979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 09:40:37.533748 11657 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 77bfd6eb3dbc4abaaadcf925c64e2bca (127.3.147.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.534439 11488 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 77bfd6eb3dbc4abaaadcf925c64e2bca (127.3.147.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.537518 11661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.537631 11658 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.537765 11658 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:37.537850 11661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:37.538375 11527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.538584 11527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:40:37.540122 11449 master.cc:552] Master@127.3.147.60:41979 shutdown complete.
I20260812 09:40:37.540262 11449 master_runner.cc:305] Clearing existing system tablet
I20260812 09:40:37.540797 11449 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:37.540940 11449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:37.543470 11449 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 09:40:37.545231 11668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:37.545609 11449 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:37.545838 11449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 09:40:37.545934 11449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
uuid: "14c4890dce4b4b68aebc5fefe3034a61"
format_stamp: "Formatted at 2026-08-12 09:40:37 on dist-test-slave-cbbz"
server_key: "3fede691b0a3263f58ec2131320f953d"
server_key_iv: "2aa71e695b47b6cc45a34b8725bcd486"
server_key_version: "encryptionkey@0"
I20260812 09:40:37.549383 11449 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:37.553966 11449 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 09:40:37.554174 11449 log.cc:1196] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 09:40:37.554720 11449 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Deleting consensus metadata
I20260812 09:40:37.555037 11449 master_runner.cc:315] Copying system tablet from 127.3.147.62:34643
I20260812 09:40:37.557758 11449 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: tablet copy: Beginning tablet copy session from remote peer at address 127.3.147.62:34643
I20260812 09:40:37.575852 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.558976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48598 (local address 127.3.147.62:34643)
0812 09:40:37.559275 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:37.559282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:37.559673 (+ 391us) server_negotiation.cc:407] Connection header received
0812 09:40:37.560033 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:37.560036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:37.560229 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:37.560406 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:37.561309 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.562464 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.564032 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.564622 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.564872 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.567550 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:37.567555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:37.567557 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:37.567601 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:37.570618 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.571951 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.571977 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.571981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.572517 (+ 536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:37.573689 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:37.573703 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:37.573707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:37.574107 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:37.574353 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:37.575049 (+ 696us) server_negotiation.cc:299] Negotiation successful
0812 09:40:37.575326 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260812 09:40:37.575959 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:37.558874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:48598)
0812 09:40:37.559438 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:37.559469 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:37.559803 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:37.560499 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:37.560503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:37.560884 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:37.561094 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.561103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.562655 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.562661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.563800 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:37.563819 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:37.564879 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:37.564894 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:37.565102 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:37.566088 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:37.566147 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:37.566990 (+ 843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:37.570852 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.570859 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.570870 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.571644 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.572976 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:37.572989 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:37.573015 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:37.573315 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:37.574353 (+ 1038us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:37.574357 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:37.574566 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 09:40:37.574847 (+ 281us) client_negotiation.cc:240] Negotiation successful
0812 09:40:37.575322 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":169,"threads_started":1}
I20260812 09:40:37.577790 10776 tablet_copy_service.cc:140] P d890b142c12c469db8922f957d18eaf6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 14c4890dce4b4b68aebc5fefe3034a61 ({username='oryx', principal='oryx/127.3.147.60@KRBTEST.COM'} at 127.0.0.1:48598)
I20260812 09:40:37.577929 10776 tablet_copy_service.cc:161] P d890b142c12c469db8922f957d18eaf6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 14c4890dce4b4b68aebc5fefe3034a61 at {username='oryx', principal='oryx/127.3.147.60@KRBTEST.COM'} at 127.0.0.1:48598: session id = 14c4890dce4b4b68aebc5fefe3034a61-00000000000000000000000000000000
I20260812 09:40:37.580308 10776 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 09:40:37.583659 11449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 09:40:37.587532 11449 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: tablet copy: Starting download of 0 data blocks...
I20260812 09:40:37.587931 11449 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: tablet copy: Starting download of 1 WAL segments...
I20260812 09:40:37.593516 11449 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 09:40:37.598112 10776 tablet_copy_service.cc:342] P d890b142c12c469db8922f957d18eaf6: Request end of tablet copy session 14c4890dce4b4b68aebc5fefe3034a61-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.147.60@KRBTEST.COM'} at 127.0.0.1:48598
I20260812 09:40:37.598369 10776 tablet_copy_service.cc:433] P d890b142c12c469db8922f957d18eaf6: ending tablet copy session 14c4890dce4b4b68aebc5fefe3034a61-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 14c4890dce4b4b68aebc5fefe3034a61
I20260812 09:40:37.611945 11449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:37.614758 11680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:37.615044 11684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:37.615674 11681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:37.615967 11449 server_base.cc:1034] running on GCE node
I20260812 09:40:37.616426 11449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:37.617036 11449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:37.618219 11449 hybrid_clock.cc:648] HybridClock initialized: now 1786527637618170 us; error 77 us; skew 500 ppm
I20260812 09:40:37.620643 11315 heartbeater.cc:503] Master 127.3.147.62:34643 was elected leader, sending a full tablet report...
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:37.621730 11449 init.cc:377] Logged in from keytab as oryx/127.3.147.60@KRBTEST.COM (short username oryx)
I20260812 09:40:37.622570 11449 webserver.cc:460] Webserver started at http://127.3.147.60:39707/ using document root <none> and password file <none>
I20260812 09:40:37.622984 11449 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:37.623113 11449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:37.625802 11449 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:37.627944 11691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:37.628892 11449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 09:40:37.629091 11449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
uuid: "14c4890dce4b4b68aebc5fefe3034a61"
format_stamp: "Formatted at 2026-08-12 09:40:37 on dist-test-slave-cbbz"
server_key: "3fede691b0a3263f58ec2131320f953d"
server_key_iv: "2aa71e695b47b6cc45a34b8725bcd486"
server_key_version: "encryptionkey@0"
I20260812 09:40:37.629307 11449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:37.647632 11449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:37.651104 11449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:37.665297 11449 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.60:41979
I20260812 09:40:37.667337 11449 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
I20260812 09:40:37.668604 11752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.60:41979 every 8 connection(s)
I20260812 09:40:37.671397 11753 sys_catalog.cc:263] Verifying existing consensus state
I20260812 09:40:37.673520 11753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Bootstrap starting.
I20260812 09:40:37.697037 11753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:40:37.697963 11753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: Bootstrap complete.
I20260812 09:40:37.698769 11753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } }
I20260812 09:40:37.698971 11753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 14c4890dce4b4b68aebc5fefe3034a61, State: Initialized, Role: LEARNER
I20260812 09:40:37.699198 11753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } }
I20260812 09:40:37.699867 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:37.700291 11753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:37.700430 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: This master's current role is: LEARNER
I20260812 09:40:37.708828 11769 catalog_manager.cc:1259] Loaded cluster ID: a6d455f680644e5aa6d7e6f9bb4b121d
I20260812 09:40:37.708978 11769 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: loading cluster ID for follower catalog manager: success
I20260812 09:40:37.712124 11769 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: acquiring CA information for follower catalog manager: success
I20260812 09:40:37.714345 11769 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 09:40:37.882236 11655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:37.883196 11655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:37.883349 11655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:37.896721 11655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:37.896914 11655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:37.897207 11655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:37.912039 11655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab
--principal=oryx/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:43785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=40893
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971,127.3.147.60:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:37.915963 11655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:37.919018 11655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:37.945494 11778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:37.945612 11776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:37.945537 11774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:37.947022 11655 server_base.cc:1034] running on GCE node
I20260812 09:40:37.947899 11655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:37.949410 11655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:37.952836 11655 hybrid_clock.cc:648] HybridClock initialized: now 1786527637952655 us; error 207 us; skew 500 ppm
Aug 12 09:40:37 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:37.961691 11655 init.cc:377] Logged in from keytab as oryx/127.3.147.1@KRBTEST.COM (short username oryx)
I20260812 09:40:37.964385 11655 webserver.cc:460] Webserver started at http://127.3.147.1:40893/ using document root <none> and password file <none>
I20260812 09:40:37.966214 11655 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:37.966544 11655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:37.977561 11655 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.007s sys 0.000s
I20260812 09:40:37.983927 11786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:37.987066 11655 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260812 09:40:37.987425 11655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "878000dad91247dab601fa220772870e"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "3ca16b7f44369269567f69724f7e9832"
server_key_iv: "5e35440c7ec01e5b720f2c8df9fbd9ca"
server_key_version: "encryptionkey@0"
I20260812 09:40:37.989162 11655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:38.022064 11655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:38.028954 11655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:38.029517 11655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:38.031487 11655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:38.034075 11655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:38.034226 11655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:38.034363 11655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:38.034497 11655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:38.047348 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.034781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.3.147.60:41979)
0812 09:40:38.035404 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.035411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.035466 (+ 55us) server_negotiation.cc:407] Connection header received
0812 09:40:38.035884 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.035889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.036062 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.036365 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:38.037390 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.040013 (+ 2623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.044242 (+ 4229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.045817 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.046083 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.046525 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.047014 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.047152 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":206,"threads_started":1}
I20260812 09:40:38.048190 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.034599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.0.0.1:34148)
0812 09:40:38.035253 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.035271 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.035530 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.036578 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.036583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.036638 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:38.037090 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.037098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.040357 (+ 3259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.040379 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.043698 (+ 3319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.043758 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.046260 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.046284 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.046617 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.046786 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.046975 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.047159 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":142,"threads_started":1}
I20260812 09:40:38.050699 11722 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 09:40:38.052227 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d890b142c12c469db8922f957d18eaf6. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:38.052613 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: This master's current role is: LEARNER
I20260812 09:40:38.080793 11655 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:43785
I20260812 09:40:38.081028 11924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:43785 every 8 connection(s)
I20260812 09:40:38.083405 11655 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:38.089749 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 11655
I20260812 09:40:38.090152 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 10992
I20260812 09:40:38.124101 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:35333
--local_ip_for_outbound_sockets=127.3.147.2
--tserver_master_addrs=127.3.147.62:34643
--webserver_port=34675
--webserver_interface=127.3.147.2
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab
--principal=oryx/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971,127.3.147.60:41979 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.60@KRBTEST.COM
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527637, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.61@KRBTEST.COM
I20260812 09:40:38.144856 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.088664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:32945 (local address 127.3.147.60:41979)
0812 09:40:38.088988 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.088996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.108470 (+ 19474us) server_negotiation.cc:407] Connection header received
0812 09:40:38.111562 (+ 3092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.111579 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.111760 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.111904 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.118199 (+ 6295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.119793 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.124820 (+ 5027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.125746 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.125959 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.135225 (+ 9266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.135232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.135237 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.135312 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.139212 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.140683 (+ 1471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.140688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.140689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.140892 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.141858 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.141866 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.141869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.142452 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.142648 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.144110 (+ 1462us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.144565 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 09:40:38.149677 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.093667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.3.147.1:32945)
0812 09:40:38.107833 (+ 14166us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.108001 (+ 168us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.110753 (+ 2752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.112781 (+ 2028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.112832 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.114958 (+ 2126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.117669 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.117759 (+ 90us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.120550 (+ 2791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.120553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.124359 (+ 3806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.124380 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.126453 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.126460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.127140 (+ 680us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.129697 (+ 2557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.129724 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.134826 (+ 5102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.139481 (+ 4655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.139499 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.139529 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.140429 (+ 900us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.141143 (+ 714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.141162 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.141165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.141573 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.142809 (+ 1236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.142817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.143491 (+ 674us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.145291 (+ 1800us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.146397 (+ 1106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13716,"mutex_wait_us":619,"thread_start_us":11307,"threads_started":1}
I20260812 09:40:38.152453 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.107886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:60725 (local address 127.3.147.62:34643)
0812 09:40:38.109361 (+ 1475us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.109378 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.110525 (+ 1147us) server_negotiation.cc:407] Connection header received
0812 09:40:38.111639 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.111643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.111790 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.111974 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.118201 (+ 6227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.119803 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.129552 (+ 9749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.130406 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.130729 (+ 323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.139134 (+ 8405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.139150 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.139153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.139251 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.144725 (+ 5474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.146815 (+ 2090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.146830 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.146852 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.147142 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.147928 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.147932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.147933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.148478 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.148746 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.151563 (+ 2817us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.151989 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1263,"thread_start_us":271,"threads_started":1}
I20260812 09:40:38.153158 11925 heartbeater.cc:347] Connected to a master server at 127.3.147.60:41979
I20260812 09:40:38.153784 11925 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.154126 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.108491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.1:60725)
0812 09:40:38.110209 (+ 1718us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.110291 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.110753 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.112781 (+ 2028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.112832 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.114961 (+ 2129us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.117669 (+ 2708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.117759 (+ 90us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.120514 (+ 2755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.120525 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.129277 (+ 8752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.129303 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.131067 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.131092 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.131497 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.134614 (+ 3117us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.134642 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.138775 (+ 4133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.145756 (+ 6981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.145759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.145761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.146479 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.147402 (+ 923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.147416 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.147418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.147549 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.149001 (+ 1452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.149004 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.149212 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.149500 (+ 288us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.153521 (+ 4021us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"mutex_wait_us":793,"spinlock_wait_cycles":65280,"thread_start_us":357,"threads_started":1}
I20260812 09:40:38.154497 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.088668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:55335 (local address 127.3.147.61:33971)
0812 09:40:38.089357 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.089386 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.109249 (+ 19863us) server_negotiation.cc:407] Connection header received
0812 09:40:38.111710 (+ 2461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.111718 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.111945 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.112191 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.118302 (+ 6111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.120180 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.124888 (+ 4708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.125694 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.125841 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.141668 (+ 15827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.141689 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.141718 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.141845 (+ 127us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.147340 (+ 5495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.149493 (+ 2153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.149502 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.149528 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.149866 (+ 338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.150907 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.150914 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.150918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.151950 (+ 1032us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.152379 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.153751 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.154114 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":206,"threads_started":1}
I20260812 09:40:38.155164 11925 heartbeater.cc:511] Master 127.3.147.60:41979 requested a full tablet report, sending...
I20260812 09:40:38.156701 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.090366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.3.147.1:55335)
0812 09:40:38.107834 (+ 17468us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.108025 (+ 191us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.110775 (+ 2750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.112796 (+ 2021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.112847 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.115011 (+ 2164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.117759 (+ 2748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.117797 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.120590 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.120618 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.124112 (+ 3494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.124360 (+ 248us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.126297 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.126301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.126764 (+ 463us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.129392 (+ 2628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.129444 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.141189 (+ 11745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.147707 (+ 6518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.147714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.147749 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.149072 (+ 1323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.150220 (+ 1148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.150249 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.150277 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.150574 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.152568 (+ 1994us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.152595 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.152968 (+ 373us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.155191 (+ 2223us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.155741 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16822,"mutex_wait_us":628,"spinlock_wait_cycles":130432,"thread_start_us":14397,"threads_started":1}
I20260812 09:40:38.158680 11926 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34643
I20260812 09:40:38.158905 11926 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.159268 11927 heartbeater.cc:347] Connected to a master server at 127.3.147.61:33971
I20260812 09:40:38.159322 11926 heartbeater.cc:511] Master 127.3.147.62:34643 requested a full tablet report, sending...
I20260812 09:40:38.159413 11927 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.159902 11927 heartbeater.cc:511] Master 127.3.147.61:33971 requested a full tablet report, sending...
I20260812 09:40:38.160614 11712 ts_manager.cc:194] Registered new tserver with Master: 878000dad91247dab601fa220772870e (127.3.147.1:43785)
I20260812 09:40:38.162343 10752 ts_manager.cc:194] Re-registered known tserver with Master: 878000dad91247dab601fa220772870e (127.3.147.1:43785)
I20260812 09:40:38.162410 11477 ts_manager.cc:194] Registered new tserver with Master: 878000dad91247dab601fa220772870e (127.3.147.1:43785)
I20260812 09:40:38.164753 10752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:60725
W20260812 09:40:38.418375 11934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:38.418987 11934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:38.419071 11934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:38.429865 11934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:38.430035 11934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:38.430322 11934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:38.442888 11934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab
--principal=oryx/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:35333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=34675
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971,127.3.147.60:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:38.446177 11934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:38.448995 11934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:38.469061 11943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:38.469061 11940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:38.469062 11939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:38.469408 11934 server_base.cc:1034] running on GCE node
I20260812 09:40:38.469851 11934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:38.470506 11934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:38.471714 11934 hybrid_clock.cc:648] HybridClock initialized: now 1786527638471695 us; error 32 us; skew 500 ppm
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527638, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:38.474538 11934 init.cc:377] Logged in from keytab as oryx/127.3.147.2@KRBTEST.COM (short username oryx)
I20260812 09:40:38.475414 11934 webserver.cc:460] Webserver started at http://127.3.147.2:34675/ using document root <none> and password file <none>
I20260812 09:40:38.476001 11934 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:38.476116 11934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:38.480049 11934 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 09:40:38.482183 11950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:38.483433 11934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:38.483587 11934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "37ddd4eba85240fbaa30d3cc8c3380fd"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "56c22a9f412df8eac6cec452076aed45"
server_key_iv: "1f2c0656bdda42e7d4b2a98ab6499dfd"
server_key_version: "encryptionkey@0"
I20260812 09:40:38.484153 11934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:38.502873 11661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: NON_VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: true } } }
I20260812 09:40:38.503053 11661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:38.503345 11934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:38.504123 11661 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: attempting to promote NON_VOTER 14c4890dce4b4b68aebc5fefe3034a61 to VOTER
I20260812 09:40:38.504580 11661 consensus_queue.cc:237] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } }
I20260812 09:40:38.505903 11722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 LEARNER]: Refusing update from remote peer d890b142c12c469db8922f957d18eaf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 09:40:38.506021 11488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Refusing update from remote peer d890b142c12c469db8922f957d18eaf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 09:40:38.506484 11955 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 09:40:38.506723 11807 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 09:40:38.507406 11934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:38.507606 11934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:38.508395 11934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:38.509395 11934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:38.509454 11934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:38.509516 11934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:38.509546 11934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:38.509658 11488 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:38.510299 11722 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:38.509228 11955 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:38.512017 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:38.512172 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:40:38.512611 11963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:38.512774 11963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:40:38.513486 11807 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:38.513612 11807 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:38.513625 11661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:38.513712 11661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:38.513643 11955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d890b142c12c469db8922f957d18eaf6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:38.513782 11955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6 [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:38.523113 11934 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:35333
I20260812 09:40:38.523198 12082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:35333 every 8 connection(s)
I20260812 09:40:38.524216 11934 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:38.526302 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 11934
I20260812 09:40:38.526537 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 11159
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527638, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.61@KRBTEST.COM
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527638, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.60@KRBTEST.COM
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527638, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:38.546775 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.526257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:35467 (local address 127.3.147.60:41979)
0812 09:40:38.526399 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.526404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.528230 (+ 1826us) server_negotiation.cc:407] Connection header received
0812 09:40:38.529646 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.529651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.529719 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.529780 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.532178 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.533316 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.535802 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.536433 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.536548 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.541558 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.541583 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.541586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.541626 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.543773 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.544550 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.544555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.544557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.544674 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.545149 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.545154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.545156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.545422 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.545508 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.546326 (+ 818us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.546494 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 09:40:38.547043 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.529097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:37619 (local address 127.3.147.62:34643)
0812 09:40:38.529226 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.529231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.529248 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:40:38.529664 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.529671 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.529747 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.529819 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.532223 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.533119 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.535715 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.536108 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.536222 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.542739 (+ 6517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.542746 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.542753 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.542802 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.544838 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.545383 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.545389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.545394 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.545495 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.545854 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.545860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.545865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.546114 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.546197 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.546618 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.546796 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:40:38.548995 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.526840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.3.147.2:35467)
0812 09:40:38.528035 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.528105 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.529367 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.530214 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.530227 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.530998 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.531959 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.531975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.533498 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.533504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.535620 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.535646 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.536856 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.536865 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.537076 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.538076 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.538100 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.541311 (+ 3211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.543941 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.543950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.543971 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.544405 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.544815 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.544818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.544821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.545016 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.545585 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.545607 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.545979 (+ 372us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.547250 (+ 1271us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.547706 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":161,"spinlock_wait_cycles":19328,"thread_start_us":810,"threads_started":1}
I20260812 09:40:38.549013 12088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.528652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.2:37619)
0812 09:40:38.529109 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.529130 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.529383 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.530216 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.530232 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.531041 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.531942 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.531965 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.533457 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.533463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.535550 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.535562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.536443 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.536447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.536873 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.538655 (+ 1782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.538671 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.542001 (+ 3330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.544971 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.544974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.544976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.545258 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.545621 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.545638 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.545643 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.545743 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.546274 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.546280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.546404 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.547252 (+ 848us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.547828 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":871,"spinlock_wait_cycles":4480,"thread_start_us":187,"threads_started":1}
I20260812 09:40:38.549770 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.526533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:50643 (local address 127.3.147.61:33971)
0812 09:40:38.526715 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.526722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.530704 (+ 3982us) server_negotiation.cc:407] Connection header received
0812 09:40:38.530922 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.530929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.530993 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.531054 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.532184 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.533129 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.535715 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.536117 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.536229 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.545439 (+ 9210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.545443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.545450 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.545491 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.547458 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.548018 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.548025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.548028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.548176 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.548518 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.548522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.548524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.548761 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.548860 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.549438 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.549589 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 09:40:38.550495 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.529322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.3.147.2:50643)
0812 09:40:38.530607 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.530625 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.530785 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.531246 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.531249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.531441 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.531961 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.531967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.533474 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.533477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.535578 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.535587 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.536443 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.536447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.536873 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.537897 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.537923 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.542536 (+ 4613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.547609 (+ 5073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.547612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.547615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.547892 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.548307 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.548311 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.548313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.548404 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.548952 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.548955 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.549157 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.549996 (+ 839us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.550205 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"mutex_wait_us":275,"spinlock_wait_cycles":1280,"thread_start_us":185,"threads_started":1}
I20260812 09:40:38.550601 12084 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34643
I20260812 09:40:38.550556 12083 heartbeater.cc:347] Connected to a master server at 127.3.147.60:41979
I20260812 09:40:38.550905 12084 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.550905 12083 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.551537 12084 heartbeater.cc:511] Master 127.3.147.62:34643 requested a full tablet report, sending...
I20260812 09:40:38.551582 12083 heartbeater.cc:511] Master 127.3.147.60:41979 requested a full tablet report, sending...
I20260812 09:40:38.552983 11712 ts_manager.cc:194] Registered new tserver with Master: 37ddd4eba85240fbaa30d3cc8c3380fd (127.3.147.2:35333)
I20260812 09:40:38.553058 10752 ts_manager.cc:194] Re-registered known tserver with Master: 37ddd4eba85240fbaa30d3cc8c3380fd (127.3.147.2:35333)
I20260812 09:40:38.553599 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:42599
--local_ip_for_outbound_sockets=127.3.147.3
--tserver_master_addrs=127.3.147.62:34643
--webserver_port=39735
--webserver_interface=127.3.147.3
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab
--principal=oryx/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971,127.3.147.60:41979 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:40:38.553928 10752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:37619
I20260812 09:40:38.555419 12085 heartbeater.cc:347] Connected to a master server at 127.3.147.61:33971
I20260812 09:40:38.555562 12085 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.555734 12085 heartbeater.cc:511] Master 127.3.147.61:33971 requested a full tablet report, sending...
I20260812 09:40:38.556521 11477 ts_manager.cc:194] Registered new tserver with Master: 37ddd4eba85240fbaa30d3cc8c3380fd (127.3.147.2:35333)
W20260812 09:40:38.684034 12090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:38.684365 12090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:38.684412 12090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:38.688735 12090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:38.688795 12090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:38.688855 12090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:38.694311 12090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab
--principal=oryx/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:42599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=39735
--tserver_master_addrs=127.3.147.62:34643,127.3.147.61:33971,127.3.147.60:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:38.695451 12090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:38.696676 12090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:38.707993 12097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:38.707988 12099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:38.708029 12096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:38.708361 12090 server_base.cc:1034] running on GCE node
I20260812 09:40:38.708938 12090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:38.709664 12090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:38.710847 12090 hybrid_clock.cc:648] HybridClock initialized: now 1786527638710817 us; error 44 us; skew 500 ppm
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527638, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:38.713690 12090 init.cc:377] Logged in from keytab as oryx/127.3.147.3@KRBTEST.COM (short username oryx)
I20260812 09:40:38.714552 12090 webserver.cc:460] Webserver started at http://127.3.147.3:39735/ using document root <none> and password file <none>
I20260812 09:40:38.715126 12090 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:38.715240 12090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:38.719408 12090 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:38.721876 12108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:38.723151 12090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:38.723310 12090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "a4f38cb3e1864666adc0c5b202e8fd9a"
format_stamp: "Formatted at 2026-08-12 09:40:36 on dist-test-slave-cbbz"
server_key: "6620a6699554e9f4e1df5b97cfaf5b1c"
server_key_iv: "0c400906d3b98beb954f7997dd36e1c2"
server_key_version: "encryptionkey@0"
I20260812 09:40:38.723938 12090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:38.758000 12090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:38.761580 12090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:38.761842 12090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:38.762565 12090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:38.763636 12090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:38.763721 12090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:38.763823 12090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:38.763882 12090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:38.781167 12090 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42599
I20260812 09:40:38.781322 12234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42599 every 8 connection(s)
I20260812 09:40:38.782413 12090 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:38.783509 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 12090
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527638, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.61@KRBTEST.COM
I20260812 09:40:38.801491 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.786093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58930 (local address 127.3.147.62:34643)
0812 09:40:38.786232 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.786236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.787378 (+ 1142us) server_negotiation.cc:407] Connection header received
0812 09:40:38.787721 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.787725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.787835 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.787951 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.789367 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.790252 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.791693 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.792119 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.792282 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.794513 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.794517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.794521 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.794552 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.796705 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.797372 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.797376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.797378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.797474 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.798738 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.798745 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.798747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.799347 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.799499 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.801132 (+ 1633us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.801249 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":66}
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527638, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
Aug 12 09:40:38 dist-test-slave-cbbz krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527638, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.60@KRBTEST.COM
I20260812 09:40:38.805004 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.786438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56886 (local address 127.3.147.61:33971)
0812 09:40:38.786644 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.786650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.788378 (+ 1728us) server_negotiation.cc:407] Connection header received
0812 09:40:38.788607 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.788613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.788679 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.788735 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.789961 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.790774 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.797210 (+ 6436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.797638 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.797761 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.799924 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.799928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.799931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.799957 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.801866 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.803339 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.803345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.803350 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.803443 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.803776 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.803781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.803786 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.804002 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.804237 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.804639 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.804795 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260812 09:40:38.806228 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.786975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.3.147.60:41979)
0812 09:40:38.787934 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.787982 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.790749 (+ 2767us) server_negotiation.cc:407] Connection header received
0812 09:40:38.790802 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.790806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.790863 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.790972 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.794295 (+ 3323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.795224 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.798764 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.799214 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.799308 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.801401 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.801404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.801406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.801450 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.803497 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.804680 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.804684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.804689 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.804786 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.805130 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.805139 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.805140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.805358 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.805472 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.805829 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.805970 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":184,"threads_started":1}
I20260812 09:40:38.806001 10752 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 09:40:38.810256 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.784809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:37989 (local address 127.3.147.60:41979)
0812 09:40:38.785984 (+ 1175us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.785993 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.786756 (+ 763us) server_negotiation.cc:407] Connection header received
0812 09:40:38.789920 (+ 3164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.789926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.789983 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.790036 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.793352 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.794103 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.797134 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.797511 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.797600 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.805861 (+ 8261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.805865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.805867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.805895 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.807790 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.808835 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.808839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.808840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.808931 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.809368 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.809372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.809374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.809603 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.809681 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.810014 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.810127 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032}
I20260812 09:40:38.810456 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.788528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:44573 (local address 127.3.147.61:33971)
0812 09:40:38.791596 (+ 3068us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.791600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.791613 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:38.791657 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.791660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.791698 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.792749 (+ 1051us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.793157 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.794070 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.797615 (+ 3545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.798037 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.798182 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.803867 (+ 5685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.803870 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.803873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.803894 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.807331 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.808065 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.808068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.808079 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.808214 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.809208 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.809214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.809217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.809446 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.809536 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.810154 (+ 618us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.810274 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3002,"thread_start_us":70,"threads_started":1}
I20260812 09:40:38.811197 12243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.785216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.3.147.3:37989)
0812 09:40:38.786602 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.786657 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.789500 (+ 2843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.790352 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.790360 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.791046 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.793175 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.793182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.795941 (+ 2759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.795944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.796966 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.796976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.797778 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.797782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.797944 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.802443 (+ 4499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.802461 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.805718 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.808372 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.808375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.808377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.808616 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.809161 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.809166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.809170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.809257 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.809738 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.809744 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.809837 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.810420 (+ 583us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.810731 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"mutex_wait_us":54,"spinlock_wait_cycles":2944,"thread_start_us":725,"threads_started":1}
I20260812 09:40:38.811277 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.787990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.3.147.3:44573)
0812 09:40:38.789543 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.789570 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.789679 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.791927 (+ 2248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.791930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.792272 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.792742 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.792752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.794369 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.794372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.796832 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.796841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.798344 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.798348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.798520 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.799503 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.799529 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.802210 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.807465 (+ 5255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.807471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.807474 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.807959 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.808977 (+ 1018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.808983 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.808989 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.809095 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.809582 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.809585 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.809677 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.809764 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.810154 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1478,"mutex_wait_us":677,"thread_start_us":283,"threads_started":1}
W20260812 09:40:38.811718 10752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:38.812539 12235 heartbeater.cc:347] Connected to a master server at 127.3.147.60:41979
I20260812 09:40:38.812732 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.786940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.3:39231)
0812 09:40:38.788667 (+ 1727us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.788686 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.789710 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.790921 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.790924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.791378 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:38.792249 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.792274 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.794362 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.794365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.796295 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.796305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.797016 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.797023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.797264 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.798957 (+ 1693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:38.798984 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:38.803285 (+ 4301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:38.807260 (+ 3975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.807266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.807282 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.807760 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.808087 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:38.808089 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:38.808091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:38.808278 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:38.808774 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:38.808780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:38.809077 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.809751 (+ 674us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.811768 (+ 2017us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1577,"mutex_wait_us":1771,"spinlock_wait_cycles":3840,"thread_start_us":113,"threads_started":1}
I20260812 09:40:38.812808 12235 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.813158 12237 heartbeater.cc:347] Connected to a master server at 127.3.147.61:33971
I20260812 09:40:38.813259 12237 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.813419 12237 heartbeater.cc:511] Master 127.3.147.61:33971 requested a full tablet report, sending...
I20260812 09:40:38.813419 12235 heartbeater.cc:511] Master 127.3.147.60:41979 requested a full tablet report, sending...
I20260812 09:40:38.809671 12247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.787014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:39231 (local address 127.3.147.62:34643)
0812 09:40:38.789995 (+ 2981us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.790001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.790017 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:40:38.790057 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.790060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.790106 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.790204 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:38.792390 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.793262 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.796412 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.796846 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.797036 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.803476 (+ 6440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:38.803480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:38.803483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:38.803506 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:38.807126 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.807893 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.807897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.807898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.807982 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:38.808391 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:38.808411 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:38.808413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:38.808607 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:38.808701 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.809355 (+ 654us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.809475 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2903,"thread_start_us":63,"threads_started":1}
I20260812 09:40:38.814599 12236 heartbeater.cc:347] Connected to a master server at 127.3.147.62:34643
I20260812 09:40:38.814713 12236 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:38.814699 11477 ts_manager.cc:194] Registered new tserver with Master: a4f38cb3e1864666adc0c5b202e8fd9a (127.3.147.3:42599)
I20260812 09:40:38.814777 11712 ts_manager.cc:194] Registered new tserver with Master: a4f38cb3e1864666adc0c5b202e8fd9a (127.3.147.3:42599)
I20260812 09:40:38.814896 12236 heartbeater.cc:511] Master 127.3.147.62:34643 requested a full tablet report, sending...
I20260812 09:40:38.815732 10751 ts_manager.cc:194] Re-registered known tserver with Master: a4f38cb3e1864666adc0c5b202e8fd9a (127.3.147.3:42599)
I20260812 09:40:38.816504 10751 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:39231
I20260812 09:40:38.833302 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.826800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.3.147.3:42599)
0812 09:40:38.827514 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.827562 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.827588 (+ 26us) server_negotiation.cc:407] Connection header received
0812 09:40:38.827673 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.827689 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.827948 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.828210 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:38.828495 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.829562 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.831220 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.832232 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.832355 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.832501 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.833062 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.833168 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":134,"threads_started":1}
I20260812 09:40:38.833274 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.826641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42599 (local address 127.0.0.1:54580)
0812 09:40:38.827028 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.827049 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.827164 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.828217 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.828220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.828242 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:38.828359 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.828365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.829855 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.829857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.831106 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.831118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.832846 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.832848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.832910 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.832940 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.833023 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.833106 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":205,"threads_started":1}
I20260812 09:40:38.834038 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.827273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:43785 (local address 127.0.0.1:41608)
0812 09:40:38.827769 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.827785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.827941 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.829322 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.829325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.829347 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:38.829440 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.829448 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.831054 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.831057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.832283 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.832294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.833605 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.833609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.833694 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.833717 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.833790 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.833861 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":41,"thread_start_us":113,"threads_started":1}
I20260812 09:40:38.834582 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.827518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.3.147.1:43785)
0812 09:40:38.828409 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.828418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.828442 (+ 24us) server_negotiation.cc:407] Connection header received
0812 09:40:38.828520 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.828525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.828733 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.829161 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:38.829623 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.830819 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.832445 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.833258 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.833376 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.833541 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.834320 (+ 779us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.834413 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":90,"threads_started":1}
I20260812 09:40:38.834815 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.827647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:35333 (local address 127.0.0.1:37578)
0812 09:40:38.827966 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.827979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.828102 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.829619 (+ 1517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.829625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.829654 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:38.829758 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.829763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.831251 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.831255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.832724 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.832734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.834325 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.834330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.834434 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.834465 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.834553 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.834616 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260812 09:40:38.836565 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.827990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.3.147.2:35333)
0812 09:40:38.828843 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.828905 (+ 62us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.828930 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:40:38.829011 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.829019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.829438 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.829637 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:38.829847 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.831153 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.832854 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.833999 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.834124 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.834282 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.836281 (+ 1999us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.836410 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":149,"threads_started":1}
I20260812 09:40:38.837159 12161 tablet_service.cc:1467] Processing CreateTablet for tablet f76135003cf84502959eceb04e118d79 (DEFAULT_TABLE table=test-table [id=34a291d8789a4e6782e1dbe02abf7a15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:38.837466 11851 tablet_service.cc:1467] Processing CreateTablet for tablet f76135003cf84502959eceb04e118d79 (DEFAULT_TABLE table=test-table [id=34a291d8789a4e6782e1dbe02abf7a15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:38.838310 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f76135003cf84502959eceb04e118d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:38.838737 11851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f76135003cf84502959eceb04e118d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:38.839709 12011 tablet_service.cc:1467] Processing CreateTablet for tablet f76135003cf84502959eceb04e118d79 (DEFAULT_TABLE table=test-table [id=34a291d8789a4e6782e1dbe02abf7a15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:38.840787 12011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f76135003cf84502959eceb04e118d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:38.847360 12270 tablet_bootstrap.cc:492] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: Bootstrap starting.
I20260812 09:40:38.848028 12272 tablet_bootstrap.cc:492] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: Bootstrap starting.
I20260812 09:40:38.849373 12273 tablet_bootstrap.cc:492] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: Bootstrap starting.
I20260812 09:40:38.851200 12272 tablet_bootstrap.cc:654] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:38.851233 12270 tablet_bootstrap.cc:654] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:38.852223 12273 tablet_bootstrap.cc:654] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:38.852984 12270 log.cc:824] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:38.852977 12272 log.cc:824] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:38.853708 12273 log.cc:824] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:38.855280 12272 tablet_bootstrap.cc:492] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: No bootstrap required, opened a new log
I20260812 09:40:38.855500 12272 ts_tablet_manager.cc:1397] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 09:40:38.856047 12270 tablet_bootstrap.cc:492] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: No bootstrap required, opened a new log
I20260812 09:40:38.856047 12273 tablet_bootstrap.cc:492] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: No bootstrap required, opened a new log
I20260812 09:40:38.856287 12273 ts_tablet_manager.cc:1397] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260812 09:40:38.856287 12270 ts_tablet_manager.cc:1397] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260812 09:40:38.860409 12272 raft_consensus.cc:348] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.860709 12272 raft_consensus.cc:374] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:38.860775 12272 raft_consensus.cc:729] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878000dad91247dab601fa220772870e, State: Initialized, Role: FOLLOWER
I20260812 09:40:38.861356 12272 consensus_queue.cc:260] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.862455 12272 ts_tablet_manager.cc:1428] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 09:40:38.862702 12273 raft_consensus.cc:348] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.863072 12273 raft_consensus.cc:374] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:38.863144 12273 raft_consensus.cc:729] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ddd4eba85240fbaa30d3cc8c3380fd, State: Initialized, Role: FOLLOWER
I20260812 09:40:38.863715 12273 consensus_queue.cc:260] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.864084 12270 raft_consensus.cc:348] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.864374 12270 raft_consensus.cc:374] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:38.864426 12270 raft_consensus.cc:729] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f38cb3e1864666adc0c5b202e8fd9a, State: Initialized, Role: FOLLOWER
I20260812 09:40:38.864861 12273 ts_tablet_manager.cc:1428] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260812 09:40:38.864847 12270 consensus_queue.cc:260] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.865021 11926 heartbeater.cc:503] Master 127.3.147.62:34643 was elected leader, sending a full tablet report...
I20260812 09:40:38.865973 12270 ts_tablet_manager.cc:1428] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260812 09:40:38.866139 12084 heartbeater.cc:503] Master 127.3.147.62:34643 was elected leader, sending a full tablet report...
I20260812 09:40:38.867869 12236 heartbeater.cc:503] Master 127.3.147.62:34643 was elected leader, sending a full tablet report...
I20260812 09:40:38.932268 12282 raft_consensus.cc:482] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:38.932581 12282 raft_consensus.cc:504] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.933925 12282 leader_election.cc:288] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4f38cb3e1864666adc0c5b202e8fd9a (127.3.147.3:42599), 878000dad91247dab601fa220772870e (127.3.147.1:43785)
I20260812 09:40:38.937853 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.934420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:48731 (local address 127.3.147.1:43785)
0812 09:40:38.934537 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.934541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.934552 (+ 11us) server_negotiation.cc:407] Connection header received
0812 09:40:38.934596 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.934598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.934644 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.934738 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:38.935091 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.935580 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.936829 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.937217 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.937336 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.937400 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.937652 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.937723 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 09:40:38.937891 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.934232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:58869 (local address 127.3.147.3:42599)
0812 09:40:38.934369 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.934374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.934391 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:40:38.934470 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.934473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.934531 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.934610 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:38.935023 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.935584 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.936854 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.937265 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.937384 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.937446 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.937705 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.937784 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:40:38.937979 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.934226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:43785 (local address 127.3.147.2:48731)
0812 09:40:38.934404 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.934418 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.934495 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.934753 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.934756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.934773 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:38.934956 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.934962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.935698 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.935705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.936696 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.936706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.937339 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.937343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.937501 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.937520 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.937612 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.937689 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":48}
I20260812 09:40:38.938041 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.934111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42599 (local address 127.3.147.2:58869)
0812 09:40:38.934254 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:38.934272 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:38.934370 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:38.934625 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:38.934628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:38.934641 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:38.934864 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.934871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.935704 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.935709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.936725 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:38.936735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.937388 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.937391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:38.937560 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.937579 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 09:40:38.937674 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 09:40:38.937746 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":48}
I20260812 09:40:38.938745 11871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f76135003cf84502959eceb04e118d79" candidate_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878000dad91247dab601fa220772870e" is_pre_election: true
I20260812 09:40:38.938824 12185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f76135003cf84502959eceb04e118d79" candidate_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" is_pre_election: true
I20260812 09:40:38.939069 11871 raft_consensus.cc:2454] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37ddd4eba85240fbaa30d3cc8c3380fd in term 0.
I20260812 09:40:38.939113 12185 raft_consensus.cc:2454] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37ddd4eba85240fbaa30d3cc8c3380fd in term 0.
I20260812 09:40:38.939574 11953 leader_election.cc:302] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37ddd4eba85240fbaa30d3cc8c3380fd, 878000dad91247dab601fa220772870e; no voters:
I20260812 09:40:38.939872 12282 raft_consensus.cc:2781] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:38.939960 12282 raft_consensus.cc:482] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:38.940004 12282 raft_consensus.cc:3037] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:38.941571 12282 raft_consensus.cc:504] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.941908 12282 leader_election.cc:288] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [CANDIDATE]: Term 1 election: Requested vote from peers a4f38cb3e1864666adc0c5b202e8fd9a (127.3.147.3:42599), 878000dad91247dab601fa220772870e (127.3.147.1:43785)
I20260812 09:40:38.942250 12185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f76135003cf84502959eceb04e118d79" candidate_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a"
I20260812 09:40:38.942303 11871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f76135003cf84502959eceb04e118d79" candidate_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878000dad91247dab601fa220772870e"
I20260812 09:40:38.942381 12185 raft_consensus.cc:3037] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:38.942449 11871 raft_consensus.cc:3037] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:38.943673 11871 raft_consensus.cc:2454] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37ddd4eba85240fbaa30d3cc8c3380fd in term 1.
I20260812 09:40:38.943943 11953 leader_election.cc:302] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37ddd4eba85240fbaa30d3cc8c3380fd, 878000dad91247dab601fa220772870e; no voters:
I20260812 09:40:38.944119 12282 raft_consensus.cc:2781] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:38.944113 12185 raft_consensus.cc:2454] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37ddd4eba85240fbaa30d3cc8c3380fd in term 1.
I20260812 09:40:38.944391 12282 raft_consensus.cc:686] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 LEADER]: Becoming Leader. State: Replica: 37ddd4eba85240fbaa30d3cc8c3380fd, State: Running, Role: LEADER
I20260812 09:40:38.944774 12282 consensus_queue.cc:237] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } }
I20260812 09:40:38.948752 10751 catalog_manager.cc:5495] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 37ddd4eba85240fbaa30d3cc8c3380fd (127.3.147.2). New cstate: current_term: 1 leader_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } health_report { overall_health: HEALTHY } } }
I20260812 09:40:38.983235 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:38.979250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.3.147.2:35333)
0812 09:40:38.979414 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:38.979417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:38.979430 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:38.979505 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:38.979507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:38.979568 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:38.979690 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:38.980100 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.980790 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.981649 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:38.981883 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:38.981997 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:38.982398 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:38.982810 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:38.982946 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:38.983004 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 09:40:38.983064 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 09:40:38.990109 11871 raft_consensus.cc:1261] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 1 FOLLOWER]: Refusing update from remote peer 37ddd4eba85240fbaa30d3cc8c3380fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:38.990109 12185 raft_consensus.cc:1261] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 FOLLOWER]: Refusing update from remote peer 37ddd4eba85240fbaa30d3cc8c3380fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:38.991015 12284 consensus_queue.cc:1035] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:38.991276 12282 consensus_queue.cc:1035] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:39.005468 12291 mvcc.cc:205] Tried to move back new op lower bound from 7317617209294925824 to 7317617209120428032. Current Snapshot: MvccSnapshot[applied={T|T < 7317617209294925824}]
I20260812 09:40:39.014612 10751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:39.014810 10751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:39.019385 10751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d890b142c12c469db8922f957d18eaf6: Sending DeleteTablet for 3 replicas of tablet f76135003cf84502959eceb04e118d79
I20260812 09:40:39.020965 12161 tablet_service.cc:1514] Processing DeleteTablet for tablet f76135003cf84502959eceb04e118d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:39 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:54580
I20260812 09:40:39.021387 11851 tablet_service.cc:1514] Processing DeleteTablet for tablet f76135003cf84502959eceb04e118d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:39 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:41608
I20260812 09:40:39.022030 12011 tablet_service.cc:1514] Processing DeleteTablet for tablet f76135003cf84502959eceb04e118d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:39 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:37578
I20260812 09:40:39.022169 12300 tablet_replica.cc:330] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: stopping tablet replica
I20260812 09:40:39.022541 12300 raft_consensus.cc:2229] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:39.022958 12300 raft_consensus.cc:2258] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:39.023291 12303 tablet_replica.cc:330] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: stopping tablet replica
I20260812 09:40:39.023624 12303 raft_consensus.cc:2229] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:39.024083 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 10717
I20260812 09:40:39.024067 12303 raft_consensus.cc:2258] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:39.027372 12300 ts_tablet_manager.cc:1905] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:39.029393 12303 ts_tablet_manager.cc:1905] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:39.031306 12300 ts_tablet_manager.cc:1918] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:39.031411 12300 log.cc:1196] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/f76135003cf84502959eceb04e118d79
I20260812 09:40:39.033377 12303 ts_tablet_manager.cc:1918] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:39.033715 12301 tablet_replica.cc:330] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: stopping tablet replica
I20260812 09:40:39.033984 12300 ts_tablet_manager.cc:1939] T f76135003cf84502959eceb04e118d79 P a4f38cb3e1864666adc0c5b202e8fd9a: Deleting consensus metadata
I20260812 09:40:39.034222 12301 raft_consensus.cc:2229] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:39.034614 12301 raft_consensus.cc:2258] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:39.036449 12301 ts_tablet_manager.cc:1905] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:39.036607 12303 log.cc:1196] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/f76135003cf84502959eceb04e118d79
I20260812 09:40:39.036998 12303 ts_tablet_manager.cc:1939] T f76135003cf84502959eceb04e118d79 P 37ddd4eba85240fbaa30d3cc8c3380fd: Deleting consensus metadata
I20260812 09:40:39.040576 12301 ts_tablet_manager.cc:1918] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:39.040807 12301 log.cc:1196] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/f76135003cf84502959eceb04e118d79
I20260812 09:40:39.041174 12301 ts_tablet_manager.cc:1939] T f76135003cf84502959eceb04e118d79 P 878000dad91247dab601fa220772870e: Deleting consensus metadata
W20260812 09:40:39.043256 11952 connection.cc:535] server connection from 127.0.0.1:37578 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 09:40:39.043660 11787 connection.cc:535] server connection from 127.0.0.1:41608 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 09:40:39.044023 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:39.023026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56896 (local address 127.3.147.61:33971)
0812 09:40:39.023164 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:39.023169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:39.031834 (+ 8665us) server_negotiation.cc:407] Connection header received
0812 09:40:39.032080 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:39.032084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:39.032160 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:39.032216 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:39.035509 (+ 3293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:39.036375 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:39.037670 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:39.038030 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:39.038104 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:39.040315 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:39.040319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:39.040322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:39.040353 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:39.042138 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:39.042641 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:39.042645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:39.042647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:39.042736 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:39.043034 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:39.043038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:39.043039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:39.043238 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:39.043339 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:39.043734 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 09:40:39.043867 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 09:40:39.044024 12111 connection.cc:535] server connection from 127.0.0.1:54580 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 09:40:39.055218 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:39.023314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.3.147.60:41979)
0812 09:40:39.023429 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:39.023434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:39.045349 (+ 21915us) server_negotiation.cc:407] Connection header received
0812 09:40:39.045450 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:39.045453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:39.045542 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:39.045650 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:39.046488 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:39.047233 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:39.049060 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:39.049470 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:39.049648 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:39.051291 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:39.051295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:39.051298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:39.051325 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:39.053117 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:39.053586 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:39.053591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:39.053594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:39.053730 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:39.054116 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:39.054123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:39.054124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:39.054353 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:39.054474 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:39.054897 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 09:40:39.055022 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":45}
I20260812 09:40:39.058758 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:39.047819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.3.147.61:33971)
0812 09:40:39.047929 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:39.047933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:39.048502 (+ 569us) server_negotiation.cc:407] Connection header received
0812 09:40:39.048655 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:39.048658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:39.048715 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:39.048760 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:39.049738 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:39.050599 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:39.052747 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:39.053088 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:39.053159 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:39.054847 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:39.054850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:39.054853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:39.054878 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:39.056996 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:39.057476 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:39.057480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:39.057481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:39.057574 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:39.057857 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:39.057860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:39.057862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:39.058062 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:39.058185 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:39.058462 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 09:40:39.058580 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 09:40:39.058894 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:39.047589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.3.147.60:41979)
0812 09:40:39.047710 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:39.047715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:39.049238 (+ 1523us) server_negotiation.cc:407] Connection header received
0812 09:40:39.049417 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:39.049420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:39.049471 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:39.049522 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:39.050610 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:39.051375 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:39.052603 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:39.052978 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:39.053093 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:39.055034 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:39.055037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:39.055040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:39.055062 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:39.057121 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:39.057610 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:39.057614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:39.057616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:39.057697 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:39.057997 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:39.058000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:39.058002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:39.058219 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:39.058350 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:39.058604 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 09:40:39.058723 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":52}
I20260812 09:40:40.004935 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.004447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.1:49699)
0812 09:40:40.004707 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.004790 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
W20260812 09:40:40.005475 11926 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:34643 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
I20260812 09:40:40.006002 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.005312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.3:55427)
0812 09:40:40.005794 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.005891 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
W20260812 09:40:40.006465 12236 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:34643 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
I20260812 09:40:40.007580 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.007132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.2:58223)
0812 09:40:40.007366 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.007474 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
W20260812 09:40:40.007817 12084 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:34643 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
I20260812 09:40:40.520361 12317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d890b142c12c469db8922f957d18eaf6)
I20260812 09:40:40.520529 12317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } }
I20260812 09:40:40.520924 12319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d890b142c12c469db8922f957d18eaf6)
I20260812 09:40:40.521185 12317 leader_election.cc:288] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d890b142c12c469db8922f957d18eaf6 (127.3.147.62:34643), 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60:41979)
I20260812 09:40:40.521137 12319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } }
I20260812 09:40:40.521885 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.521307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:59384)
0812 09:40:40.521646 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.521735 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":95,"threads_started":1}
W20260812 09:40:40.522143 11465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
I20260812 09:40:40.522341 12319 leader_election.cc:288] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d890b142c12c469db8922f957d18eaf6 (127.3.147.62:34643), 77bfd6eb3dbc4abaaadcf925c64e2bca (127.3.147.61:33971)
I20260812 09:40:40.522939 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.522386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:59390)
0812 09:40:40.522759 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.522836 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":63,"threads_started":1}
W20260812 09:40:40.523098 11695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
I20260812 09:40:40.523598 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.523351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:59394)
0812 09:40:40.523481 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.523531 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 09:40:40.523744 11465 leader_election.cc:334] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d890b142c12c469db8922f957d18eaf6 (127.3.147.62:34643): Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
I20260812 09:40:40.524107 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.523686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:59408)
0812 09:40:40.523997 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.524045 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257}
W20260812 09:40:40.524303 11695 leader_election.cc:334] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d890b142c12c469db8922f957d18eaf6 (127.3.147.62:34643): Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
I20260812 09:40:40.526783 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.521885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.3.147.60:41979)
0812 09:40:40.522765 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:40.522771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:40.522786 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:40:40.522831 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:40.522834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:40.522880 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:40.523740 (+ 860us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:40.523851 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.524632 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.525775 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.526273 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.526368 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.526418 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:40.526629 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 09:40:40.526694 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":832,"thread_start_us":70,"threads_started":1}
I20260812 09:40:40.526821 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.521723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:41979 (local address 127.0.0.1:48792)
0812 09:40:40.522302 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.522317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:40.522423 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:40.523460 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:40.523466 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:40.523488 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:40.523624 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.523632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.524749 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.524752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.525626 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.525646 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.526398 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.526402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.526493 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.526518 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 09:40:40.526602 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 09:40:40.526697 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":92,"threads_started":1}
I20260812 09:40:40.527329 11722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "14c4890dce4b4b68aebc5fefe3034a61" is_pre_election: true
I20260812 09:40:40.527537 11722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77bfd6eb3dbc4abaaadcf925c64e2bca in term 1.
I20260812 09:40:40.527877 11465 leader_election.cc:302] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 14c4890dce4b4b68aebc5fefe3034a61, 77bfd6eb3dbc4abaaadcf925c64e2bca; no voters: d890b142c12c469db8922f957d18eaf6
I20260812 09:40:40.527958 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.522658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33971 (local address 127.0.0.1:56914)
0812 09:40:40.523092 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.523106 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:40.523190 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:40.524293 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:40.524296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:40.524317 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:40.524419 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.524426 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.525549 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.525552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.526625 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.526636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.527475 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.527481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.527580 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.527615 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 09:40:40.527736 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 09:40:40.527811 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":65,"thread_start_us":81,"threads_started":1}
I20260812 09:40:40.528000 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.522981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.3.147.61:33971)
0812 09:40:40.523972 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:40.523983 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:40.523998 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:40:40.524041 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:40.524044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:40.524110 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:40.524525 (+ 415us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:40.524625 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.525419 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.526752 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.527330 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.527446 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.527501 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:40.527752 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 09:40:40.527832 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":123,"threads_started":1}
I20260812 09:40:40.528349 12317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 09:40:40.528425 12317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Starting leader election (detected failure of leader d890b142c12c469db8922f957d18eaf6)
I20260812 09:40:40.528496 12317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 1 FOLLOWER]: Advancing to term 2
I20260812 09:40:40.528445 11488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14c4890dce4b4b68aebc5fefe3034a61" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" is_pre_election: true
I20260812 09:40:40.529706 12317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } }
I20260812 09:40:40.529968 11488 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 14c4890dce4b4b68aebc5fefe3034a61 in current term 2: Already voted for candidate 77bfd6eb3dbc4abaaadcf925c64e2bca in this term.
I20260812 09:40:40.530442 12317 leader_election.cc:288] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [CANDIDATE]: Term 2 election: Requested vote from peers d890b142c12c469db8922f957d18eaf6 (127.3.147.62:34643), 14c4890dce4b4b68aebc5fefe3034a61 (127.3.147.60:41979)
I20260812 09:40:40.530622 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.530218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:59424)
0812 09:40:40.530392 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.530466 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 09:40:40.530776 11722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "14c4890dce4b4b68aebc5fefe3034a61"
I20260812 09:40:40.530792 11694 leader_election.cc:302] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14c4890dce4b4b68aebc5fefe3034a61; no voters: 77bfd6eb3dbc4abaaadcf925c64e2bca, d890b142c12c469db8922f957d18eaf6
I20260812 09:40:40.530896 11722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 1 FOLLOWER]: Advancing to term 2
I20260812 09:40:40.531342 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.531122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:59426)
0812 09:40:40.531231 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.531283 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 09:40:40.531459 11465 leader_election.cc:334] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d890b142c12c469db8922f957d18eaf6 (127.3.147.62:34643): Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
I20260812 09:40:40.532015 11722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77bfd6eb3dbc4abaaadcf925c64e2bca in term 2.
I20260812 09:40:40.532191 12319 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:40:40.532352 11465 leader_election.cc:302] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 14c4890dce4b4b68aebc5fefe3034a61, 77bfd6eb3dbc4abaaadcf925c64e2bca; no voters: d890b142c12c469db8922f957d18eaf6
I20260812 09:40:40.532538 12317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 2 FOLLOWER]: Leader election won for term 2
I20260812 09:40:40.532783 12317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [term 2 LEADER]: Becoming Leader. State: Replica: 77bfd6eb3dbc4abaaadcf925c64e2bca, State: Running, Role: LEADER
I20260812 09:40:40.533030 12317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } }
I20260812 09:40:40.533928 12328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77bfd6eb3dbc4abaaadcf925c64e2bca. Latest consensus state: current_term: 2 leader_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:40.534053 12328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:40.534348 12330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:40.536847 12330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:40.537261 12330 catalog_manager.cc:1259] Loaded cluster ID: a6d455f680644e5aa6d7e6f9bb4b121d
I20260812 09:40:40.537304 12330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:40.537873 12330 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:40.538265 12330 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Loaded TSK: 0
I20260812 09:40:40.538932 12330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:40:40.587551 11477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:40.591079 11477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:40.594238 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.593959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:59438)
0812 09:40:40.594099 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.594168 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 09:40:40.594297 11722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [term 2 FOLLOWER]: Refusing update from remote peer 77bfd6eb3dbc4abaaadcf925c64e2bca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 09:40:40.595014 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.594816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.0.0.1:59450)
0812 09:40:40.594914 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.594960 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 09:40:40.595038 12328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 09:40:40.595253 11465 consensus_peers.cc:487] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca -> Peer d890b142c12c469db8922f957d18eaf6 (127.3.147.62:34643): Couldn't send request to peer d890b142c12c469db8922f957d18eaf6. Status: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 09:40:40.596738 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77bfd6eb3dbc4abaaadcf925c64e2bca. Latest consensus state: current_term: 2 leader_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:40.596884 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:40:40.597944 12328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:40.598086 12328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:40.598541 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d890b142c12c469db8922f957d18eaf6" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 34643 } } peers { permanent_uuid: "77bfd6eb3dbc4abaaadcf925c64e2bca" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33971 } attrs { promote: false } } peers { permanent_uuid: "14c4890dce4b4b68aebc5fefe3034a61" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 41979 } attrs { promote: false } } }
I20260812 09:40:40.598680 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c4890dce4b4b68aebc5fefe3034a61 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:40:40.608662 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.604343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42599 (local address 127.0.0.1:54592)
0812 09:40:40.604868 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.604881 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:40.604947 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:40.605183 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:40.605186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:40.605204 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:40.605300 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.605306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.606231 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.606237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.607290 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.607298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.608282 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.608286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.608378 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.608407 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 09:40:40.608453 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 09:40:40.608521 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":2}
I20260812 09:40:40.609396 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.604603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.3.147.3:42599)
0812 09:40:40.604953 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:40.604957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:40.604971 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:40.605022 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:40.605024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:40.605066 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:40.605174 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:40.605430 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.606112 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.607444 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.608096 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.609068 (+ 972us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.609152 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:40.609190 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 09:40:40.609263 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":95,"threads_started":1}
I20260812 09:40:40.609668 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.605744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.3.147.1:43785)
0812 09:40:40.606017 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:40.606022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:40.606036 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:40.606091 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:40.606094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:40.606139 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:40.606273 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:40.606582 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.607368 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.608549 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.609111 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.609236 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.609300 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:40.609464 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 09:40:40.609543 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":90,"threads_started":1}
I20260812 09:40:40.609917 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.605368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:43785 (local address 127.0.0.1:41624)
0812 09:40:40.605593 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.605608 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:40.605744 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:40.606280 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:40.606283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:40.606303 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:40.606444 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.606450 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.607506 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.607510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.608430 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.608440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.609237 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.609240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.609328 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.609350 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 09:40:40.609445 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 09:40:40.609514 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":42,"thread_start_us":88,"threads_started":1}
I20260812 09:40:40.610136 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.605168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:35333 (local address 127.0.0.1:37592)
0812 09:40:40.605488 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.605504 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:40.605579 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:40.606304 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:40.606307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:40.606325 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:40.606455 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.606461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.607563 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.607568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.608753 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.608764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.609700 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.609706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.609818 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.609843 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 09:40:40.609901 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 09:40:40.609956 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":67}
I20260812 09:40:40.610117 12161 tablet_service.cc:1467] Processing CreateTablet for tablet 03c553ac454a420a885e06e8d5f535a1 (DEFAULT_TABLE table=test-table [id=4e838c0447dc4e6a928dd2636e5cb895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:40.610430 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c553ac454a420a885e06e8d5f535a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:40.610770 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.605283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.3.147.2:35333)
0812 09:40:40.606040 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:40.606045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:40.606061 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:40:40.606111 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:40.606113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:40.606177 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:40.606588 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:40.606701 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.607449 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.608912 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.609542 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.610471 (+ 929us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.610548 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:40.610594 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 09:40:40.610670 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":87,"threads_started":1}
I20260812 09:40:40.612809 12342 tablet_bootstrap.cc:492] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: Bootstrap starting.
I20260812 09:40:40.613397 11851 tablet_service.cc:1467] Processing CreateTablet for tablet 03c553ac454a420a885e06e8d5f535a1 (DEFAULT_TABLE table=test-table [id=4e838c0447dc4e6a928dd2636e5cb895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:40.613554 12011 tablet_service.cc:1467] Processing CreateTablet for tablet 03c553ac454a420a885e06e8d5f535a1 (DEFAULT_TABLE table=test-table [id=4e838c0447dc4e6a928dd2636e5cb895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:40.613781 11851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c553ac454a420a885e06e8d5f535a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:40.613857 12342 tablet_bootstrap.cc:654] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:40.613849 12011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c553ac454a420a885e06e8d5f535a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:40.615949 12342 tablet_bootstrap.cc:492] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: No bootstrap required, opened a new log
I20260812 09:40:40.616052 12342 ts_tablet_manager.cc:1397] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:40.616317 12345 tablet_bootstrap.cc:492] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e: Bootstrap starting.
I20260812 09:40:40.616765 12346 tablet_bootstrap.cc:492] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: Bootstrap starting.
I20260812 09:40:40.616791 12342 raft_consensus.cc:348] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.616910 12342 raft_consensus.cc:374] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:40.616946 12342 raft_consensus.cc:729] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f38cb3e1864666adc0c5b202e8fd9a, State: Initialized, Role: FOLLOWER
I20260812 09:40:40.617075 12342 consensus_queue.cc:260] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.617312 12345 tablet_bootstrap.cc:654] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:40.617455 12342 ts_tablet_manager.cc:1428] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 09:40:40.617763 12346 tablet_bootstrap.cc:654] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:40.618844 12345 tablet_bootstrap.cc:492] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e: No bootstrap required, opened a new log
I20260812 09:40:40.619151 12346 tablet_bootstrap.cc:492] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: No bootstrap required, opened a new log
I20260812 09:40:40.619293 12346 ts_tablet_manager.cc:1397] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:40.618949 12345 ts_tablet_manager.cc:1397] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:40.619814 12346 raft_consensus.cc:348] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.620044 12346 raft_consensus.cc:374] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:40.620126 12346 raft_consensus.cc:729] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ddd4eba85240fbaa30d3cc8c3380fd, State: Initialized, Role: FOLLOWER
I20260812 09:40:40.620369 12345 raft_consensus.cc:348] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.620379 12346 consensus_queue.cc:260] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.620481 12345 raft_consensus.cc:374] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:40.620501 12237 heartbeater.cc:503] Master 127.3.147.61:33971 was elected leader, sending a full tablet report...
I20260812 09:40:40.620520 12345 raft_consensus.cc:729] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878000dad91247dab601fa220772870e, State: Initialized, Role: FOLLOWER
I20260812 09:40:40.620533 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.617907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.3:54661)
0812 09:40:40.620348 (+ 2441us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.620418 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2368,"thread_start_us":97,"threads_started":1}
I20260812 09:40:40.620680 12345 consensus_queue.cc:260] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.620995 12346 ts_tablet_manager.cc:1428] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 09:40:40.621029 12345 ts_tablet_manager.cc:1428] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:40.621399 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.620969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.3:58077)
0812 09:40:40.621257 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.621313 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":105,"threads_started":1}
I20260812 09:40:40.622920 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.622423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.1:44507)
0812 09:40:40.622761 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.622823 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":110,"threads_started":1}
I20260812 09:40:40.623172 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.622045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.2:38365)
0812 09:40:40.623007 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.623065 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":884,"thread_start_us":98,"threads_started":1}
I20260812 09:40:40.623998 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.623615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.2:39989)
0812 09:40:40.623853 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.623907 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
I20260812 09:40:40.624240 12085 heartbeater.cc:503] Master 127.3.147.61:33971 was elected leader, sending a full tablet report...
I20260812 09:40:40.625201 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.624697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.1:47291)
0812 09:40:40.624993 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.625125 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
I20260812 09:40:40.625517 11927 heartbeater.cc:503] Master 127.3.147.61:33971 was elected leader, sending a full tablet report...
W20260812 09:40:40.781860 12086 tablet.cc:2369] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:40:40.785956 12238 tablet.cc:2369] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:40:40.809314 12347 raft_consensus.cc:482] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:40.809494 12347 raft_consensus.cc:504] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.810664 12347 leader_election.cc:288] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37ddd4eba85240fbaa30d3cc8c3380fd (127.3.147.2:35333), 878000dad91247dab601fa220772870e (127.3.147.1:43785)
I20260812 09:40:40.814607 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.810954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:35333 (local address 127.3.147.3:54139)
0812 09:40:40.811127 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.811154 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:40.811320 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:40.811614 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:40.811616 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:40.811629 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:40.811770 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.811777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.812605 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.812610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.813450 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.813459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.814116 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.814120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.814251 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.814269 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 09:40:40.814362 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 09:40:40.814435 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":1}
I20260812 09:40:40.814594 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.811085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:54139 (local address 127.3.147.2:35333)
0812 09:40:40.811253 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:40.811259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:40.811279 (+ 20us) server_negotiation.cc:407] Connection header received
0812 09:40:40.811468 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:40.811471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:40.811528 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:40.811596 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:40.811893 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.812491 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.813585 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.813988 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.814112 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.814178 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:40.814382 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 09:40:40.814457 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 09:40:40.814643 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.811388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:45765 (local address 127.3.147.1:43785)
0812 09:40:40.811518 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:40.811522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:40.811533 (+ 11us) server_negotiation.cc:407] Connection header received
0812 09:40:40.811579 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:40.811581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:40.811622 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:40.811711 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:40.811993 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.812618 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.813638 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.814043 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.814176 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.814241 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:40.814438 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 09:40:40.814515 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:40:40.814643 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.811244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:43785 (local address 127.3.147.3:45765)
0812 09:40:40.811390 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.811405 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:40.811492 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:40.811721 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:40.811724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:40.811740 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:40.811876 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.811882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.812733 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.812737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.813504 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:40.813512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.814203 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.814207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:40.814312 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.814330 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 09:40:40.814419 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 09:40:40.814491 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":20}
I20260812 09:40:40.815122 11871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c553ac454a420a885e06e8d5f535a1" candidate_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878000dad91247dab601fa220772870e" is_pre_election: true
I20260812 09:40:40.815201 12032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c553ac454a420a885e06e8d5f535a1" candidate_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" is_pre_election: true
I20260812 09:40:40.815287 11871 raft_consensus.cc:2454] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4f38cb3e1864666adc0c5b202e8fd9a in term 0.
I20260812 09:40:40.815433 12032 raft_consensus.cc:2454] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4f38cb3e1864666adc0c5b202e8fd9a in term 0.
I20260812 09:40:40.815740 12112 leader_election.cc:302] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878000dad91247dab601fa220772870e, a4f38cb3e1864666adc0c5b202e8fd9a; no voters:
I20260812 09:40:40.815984 12347 raft_consensus.cc:2781] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:40.816061 12347 raft_consensus.cc:482] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:40.816120 12347 raft_consensus.cc:3037] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:40.817272 12347 raft_consensus.cc:504] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.817610 12347 leader_election.cc:288] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [CANDIDATE]: Term 1 election: Requested vote from peers 37ddd4eba85240fbaa30d3cc8c3380fd (127.3.147.2:35333), 878000dad91247dab601fa220772870e (127.3.147.1:43785)
I20260812 09:40:40.817977 12032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c553ac454a420a885e06e8d5f535a1" candidate_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd"
I20260812 09:40:40.818066 11871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c553ac454a420a885e06e8d5f535a1" candidate_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878000dad91247dab601fa220772870e"
I20260812 09:40:40.818158 12032 raft_consensus.cc:3037] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:40.818186 11871 raft_consensus.cc:3037] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:40.819278 11871 raft_consensus.cc:2454] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4f38cb3e1864666adc0c5b202e8fd9a in term 1.
I20260812 09:40:40.819308 12032 raft_consensus.cc:2454] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4f38cb3e1864666adc0c5b202e8fd9a in term 1.
I20260812 09:40:40.819571 12112 leader_election.cc:302] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878000dad91247dab601fa220772870e, a4f38cb3e1864666adc0c5b202e8fd9a; no voters:
I20260812 09:40:40.819813 12347 raft_consensus.cc:2781] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:40.820050 12347 raft_consensus.cc:686] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 LEADER]: Becoming Leader. State: Replica: a4f38cb3e1864666adc0c5b202e8fd9a, State: Running, Role: LEADER
I20260812 09:40:40.820338 12347 consensus_queue.cc:237] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } }
I20260812 09:40:40.820798 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.820554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.3:43105)
0812 09:40:40.820663 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.820721 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 09:40:40.823246 11477 catalog_manager.cc:5495] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a reported cstate change: term changed from 0 to 1, leader changed from <none> to a4f38cb3e1864666adc0c5b202e8fd9a (127.3.147.3). New cstate: current_term: 1 leader_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f38cb3e1864666adc0c5b202e8fd9a" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 } health_report { overall_health: UNKNOWN } } }
I20260812 09:40:40.844000 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.840455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54602 (local address 127.3.147.3:42599)
0812 09:40:40.840592 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:40.840598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:40.840612 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:40.840806 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:40.840809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:40.840864 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:40.840967 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:40.841224 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.841722 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.842571 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:40.842794 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:40.842902 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:40.843217 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:40.843630 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:40.843760 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:40.843804 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 09:40:40.843858 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 09:40:40.847424 11871 raft_consensus.cc:1261] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 1 FOLLOWER]: Refusing update from remote peer a4f38cb3e1864666adc0c5b202e8fd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:40.847559 12032 raft_consensus.cc:1261] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 FOLLOWER]: Refusing update from remote peer a4f38cb3e1864666adc0c5b202e8fd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:40.848094 12360 consensus_queue.cc:1035] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "878000dad91247dab601fa220772870e" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:40.848320 12347 consensus_queue.cc:1035] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ddd4eba85240fbaa30d3cc8c3380fd" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 35333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:40.851512 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.849851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.1:45019)
0812 09:40:40.851358 (+ 1507us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.851429 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1425}
I20260812 09:40:40.851958 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.849641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.2:60425)
0812 09:40:40.851836 (+ 2195us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.851889 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2125}
I20260812 09:40:40.852851 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.852501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.3:58459)
0812 09:40:40.852728 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.852782 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260812 09:40:40.853327 12365 mvcc.cc:205] Tried to move back new op lower bound from 7317617216904335360 to 7317617216803069952. Current Snapshot: MvccSnapshot[applied={T|T < 7317617216904335360}]
I20260812 09:40:40.854452 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.854222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.1:39507)
0812 09:40:40.854362 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.854402 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 09:40:40.856535 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.856300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.3:60317)
0812 09:40:40.856405 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.856470 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 09:40:40.858085 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.857825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:34643 (local address 127.3.147.2:57071)
0812 09:40:40.857938 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.858004 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 09:40:40.864362 11477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:40.864537 11477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:40.868788 11477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77bfd6eb3dbc4abaaadcf925c64e2bca: Sending DeleteTablet for 3 replicas of tablet 03c553ac454a420a885e06e8d5f535a1
I20260812 09:40:40.869522 12161 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c553ac454a420a885e06e8d5f535a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:40 UTC) from {username='oryx', principal='oryx/127.3.147.61@KRBTEST.COM'} at 127.0.0.1:54592
I20260812 09:40:40.869958 12376 tablet_replica.cc:330] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: stopping tablet replica
I20260812 09:40:40.870117 12376 raft_consensus.cc:2229] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:40.870462 12376 raft_consensus.cc:2258] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:40.870522 11851 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c553ac454a420a885e06e8d5f535a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:40 UTC) from {username='oryx', principal='oryx/127.3.147.61@KRBTEST.COM'} at 127.0.0.1:41624
I20260812 09:40:40.870884 12011 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c553ac454a420a885e06e8d5f535a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:40 UTC) from {username='oryx', principal='oryx/127.3.147.61@KRBTEST.COM'} at 127.0.0.1:37592
I20260812 09:40:40.870924 12377 tablet_replica.cc:330] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e: stopping tablet replica
I20260812 09:40:40.871076 12377 raft_consensus.cc:2229] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:40.871269 12377 raft_consensus.cc:2258] T 03c553ac454a420a885e06e8d5f535a1 P 878000dad91247dab601fa220772870e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:40.871294 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 11655
I20260812 09:40:40.871392 12378 tablet_replica.cc:330] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: stopping tablet replica
I20260812 09:40:40.871600 12378 raft_consensus.cc:2229] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:40.871879 12378 raft_consensus.cc:2258] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:40.873421 12378 ts_tablet_manager.cc:1905] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:40.875903 12378 ts_tablet_manager.cc:1918] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:40.876022 12378 log.cc:1196] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/03c553ac454a420a885e06e8d5f535a1
I20260812 09:40:40.876410 12378 ts_tablet_manager.cc:1939] T 03c553ac454a420a885e06e8d5f535a1 P 37ddd4eba85240fbaa30d3cc8c3380fd: Deleting consensus metadata
I20260812 09:40:40.877357 11465 catalog_manager.cc:4841] TS 37ddd4eba85240fbaa30d3cc8c3380fd (127.3.147.2:35333): tablet 03c553ac454a420a885e06e8d5f535a1 (table test-table [id=4e838c0447dc4e6a928dd2636e5cb895]) successfully deleted
I20260812 09:40:40.880769 12376 ts_tablet_manager.cc:1905] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:40.882822 12376 ts_tablet_manager.cc:1918] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:40.882917 12376 log.cc:1196] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/03c553ac454a420a885e06e8d5f535a1
I20260812 09:40:40.883173 12376 ts_tablet_manager.cc:1939] T 03c553ac454a420a885e06e8d5f535a1 P a4f38cb3e1864666adc0c5b202e8fd9a: Deleting consensus metadata
I20260812 09:40:40.884004 11466 catalog_manager.cc:4841] TS a4f38cb3e1864666adc0c5b202e8fd9a (127.3.147.3:42599): tablet 03c553ac454a420a885e06e8d5f535a1 (table test-table [id=4e838c0447dc4e6a928dd2636e5cb895]) successfully deleted
I20260812 09:40:40.885880 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 11934
I20260812 09:40:40.888520 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:40.886079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:43785 (local address 127.0.0.1:41634)
0812 09:40:40.888286 (+ 2207us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:40.888400 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2037}
W20260812 09:40:40.890353 11465 catalog_manager.cc:4584] TS 878000dad91247dab601fa220772870e (127.3.147.1:43785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 03c553ac454a420a885e06e8d5f535a1: Network error: Client connection negotiation failed: client connection to 127.3.147.1:43785: connect: Connection refused (error 111)
I20260812 09:40:40.897464 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 12090
I20260812 09:40:40.905700 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 11321
I20260812 09:40:40.916999 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 11449
2026-08-12T09:40:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6581 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:40 dist-test-slave-cbbz krb5kdc[12381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:40 dist-test-slave-cbbz krb5kdc[12381](info): set up 2 sockets
Aug 12 09:40:40 dist-test-slave-cbbz krb5kdc[12381](info): commencing operation
krb5kdc: starting...
W20260812 09:40:42.042475 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:42 dist-test-slave-cbbz krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.62.keytab.
Entry for principal oryx/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.62.keytab.
I20260812 09:40:42.167562 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:40855
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.62.keytab
--principal=oryx/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:42.282781 12396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:42.283074 12396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:42.283118 12396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:42.287132 12396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:42.287189 12396 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 09:40:42.287210 12396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:42.287231 12396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:42.287251 12396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:42.287269 12396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:42.292481 12396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:40855
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.62.keytab
--principal=oryx/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:40855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:42.293408 12396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:42.294386 12396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:42.303200 12401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:42.303161 12402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:42.303315 12396 server_base.cc:1034] running on GCE node
W20260812 09:40:42.303243 12405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:42.304015 12396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:42.305102 12396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:42.306280 12396 hybrid_clock.cc:648] HybridClock initialized: now 1786527642306253 us; error 47 us; skew 500 ppm
Aug 12 09:40:42 dist-test-slave-cbbz krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527642, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:42.309016 12396 init.cc:377] Logged in from keytab as oryx/127.3.147.62@KRBTEST.COM (short username oryx)
I20260812 09:40:42.309921 12396 webserver.cc:460] Webserver started at http://127.3.147.62:32973/ using document root <none> and password file <none>
I20260812 09:40:42.310483 12396 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:42.310576 12396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:42.310810 12396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:42.312891 12396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "681a506d6a124c0b89626253b922599a"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "8be91b05c8f7de1d1919f9bb310812ee"
server_key_iv: "8678a7f24df8027e5057472b26e20031"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.313475 12396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "681a506d6a124c0b89626253b922599a"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "8be91b05c8f7de1d1919f9bb310812ee"
server_key_iv: "8678a7f24df8027e5057472b26e20031"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.317248 12396 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:40:42.319731 12413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:42.320956 12396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:42.321096 12396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "681a506d6a124c0b89626253b922599a"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "8be91b05c8f7de1d1919f9bb310812ee"
server_key_iv: "8678a7f24df8027e5057472b26e20031"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.321209 12396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:42.333079 12396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:42.336189 12396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:42.336408 12396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:42.344637 12396 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:40855
I20260812 09:40:42.344667 12483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:40855 every 8 connection(s)
I20260812 09:40:42.345625 12396 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:42.348459 12484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:42.354100 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 12396
I20260812 09:40:42.354262 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:42.354326 12484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: Bootstrap starting.
I20260812 09:40:42.354599 3660 external_mini_cluster.cc:1428] Setting key a1c3312fe2ddf4373333d3911b2238c4
I20260812 09:40:42.357146 12484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:42.357929 12484 log.cc:824] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:42.359848 12484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: No bootstrap required, opened a new log
Aug 12 09:40:42 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:42.364602 12484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681a506d6a124c0b89626253b922599a" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40855 } }
I20260812 09:40:42.364936 12484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:42.365006 12484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681a506d6a124c0b89626253b922599a, State: Initialized, Role: FOLLOWER
I20260812 09:40:42.365484 12484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681a506d6a124c0b89626253b922599a" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40855 } }
I20260812 09:40:42.365644 12484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:42.365743 12484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:42.365850 12484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:42.367035 12484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681a506d6a124c0b89626253b922599a" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40855 } }
I20260812 09:40:42.367388 12484 leader_election.cc:302] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 681a506d6a124c0b89626253b922599a; no voters:
I20260812 09:40:42.367817 12484 leader_election.cc:288] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:42.367913 12492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:42.368127 12492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [term 1 LEADER]: Becoming Leader. State: Replica: 681a506d6a124c0b89626253b922599a, State: Running, Role: LEADER
I20260812 09:40:42.368482 12492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681a506d6a124c0b89626253b922599a" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40855 } }
I20260812 09:40:42.369297 12484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:42.369727 12492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 681a506d6a124c0b89626253b922599a. Latest consensus state: current_term: 1 leader_uuid: "681a506d6a124c0b89626253b922599a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681a506d6a124c0b89626253b922599a" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40855 } } }
I20260812 09:40:42.369843 12492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:42.370210 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.356076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.3.147.62:40855)
0812 09:40:42.356581 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:42.356597 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:42.356633 (+ 36us) server_negotiation.cc:407] Connection header received
0812 09:40:42.357379 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:42.357400 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:42.357731 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:42.358082 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:42.358946 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.359922 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.361127 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.361600 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.361720 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.364607 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:42.364616 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:42.364635 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:42.364668 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:42.366811 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.367272 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.367285 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.367290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.367407 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.367720 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.367726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.367739 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.368342 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:42.368578 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:42.368890 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 09:40:42.369189 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":150,"threads_started":1}
I20260812 09:40:42.370527 12493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "681a506d6a124c0b89626253b922599a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681a506d6a124c0b89626253b922599a" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40855 } } }
I20260812 09:40:42.370618 12493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:42.370918 12496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:42.374542 12496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:42.380913 12496 catalog_manager.cc:1347] Generated new cluster ID: e1857b2aa540407d9a4c3fa7ed64da99
I20260812 09:40:42.381033 12496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:42.396171 12496 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:42.397085 12496 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:42.402908 12496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: Generated new TSK 0
I20260812 09:40:42.403587 12496 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab.
Entry for principal oryx/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab.
I20260812 09:40:42.438871 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40855
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab
--principal=oryx/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:42.568037 12519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:42.568425 12519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:42.568503 12519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:42.572960 12519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:42.573010 12519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:42.573047 12519 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:42.573161 12519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:42.578289 12519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab
--principal=oryx/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:42.579253 12519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:42.580263 12519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:42.589586 12525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:42.589627 12524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:42.589686 12527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:42.589823 12519 server_base.cc:1034] running on GCE node
I20260812 09:40:42.590260 12519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:42.590871 12519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:42.592033 12519 hybrid_clock.cc:648] HybridClock initialized: now 1786527642592006 us; error 39 us; skew 500 ppm
Aug 12 09:40:42 dist-test-slave-cbbz krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527642, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:42.594789 12519 init.cc:377] Logged in from keytab as oryx/127.3.147.1@KRBTEST.COM (short username oryx)
I20260812 09:40:42.595739 12519 webserver.cc:460] Webserver started at http://127.3.147.1:37303/ using document root <none> and password file <none>
I20260812 09:40:42.596360 12519 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:42.596432 12519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:42.596681 12519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:42.598711 12519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "35d9aa44630d479aa34ef0db5965ccb8"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "b01ee6c591ac59ffca6f7d9e88408fed"
server_key_iv: "a00c075f6ec81a610e4c6d2cdd487c94"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.599298 12519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "35d9aa44630d479aa34ef0db5965ccb8"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "b01ee6c591ac59ffca6f7d9e88408fed"
server_key_iv: "a00c075f6ec81a610e4c6d2cdd487c94"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.603164 12519 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 09:40:42.605756 12534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:42.606950 12519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:42.607105 12519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "35d9aa44630d479aa34ef0db5965ccb8"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "b01ee6c591ac59ffca6f7d9e88408fed"
server_key_iv: "a00c075f6ec81a610e4c6d2cdd487c94"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.607213 12519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:42.622258 12519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:42.625365 12519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:42.625600 12519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:42.626945 12519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:42.627019 12519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:42.627090 12519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:42.627260 12519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:42 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527642, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:42.639986 12519 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:36963
I20260812 09:40:42.641197 12519 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:42.642398 12670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:36963 every 8 connection(s)
I20260812 09:40:42.642930 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.628883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:55447 (local address 127.3.147.62:40855)
0812 09:40:42.629035 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:42.629040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:42.629966 (+ 926us) server_negotiation.cc:407] Connection header received
0812 09:40:42.631509 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:42.631515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:42.631580 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:42.631685 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:42.633071 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.633608 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.634736 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.634982 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.635097 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.638257 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:42.638261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:42.638264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:42.638298 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:42.640248 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.640925 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.640928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.640930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.641027 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.641467 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.641470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.641472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.641755 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:42.641867 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:42.642601 (+ 734us) server_negotiation.cc:299] Negotiation successful
0812 09:40:42.642753 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 09:40:42.643963 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.629262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40855 (local address 127.3.147.1:55447)
0812 09:40:42.629778 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:42.629836 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:42.631183 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:42.631858 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:42.631865 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:42.632294 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:42.632916 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:42.632932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.633739 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.633742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:42.634598 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:42.634608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.635154 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.635159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:42.635320 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.635928 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:42.635949 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:42.638092 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:42.640414 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:42.640420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:42.640433 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:42.640789 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:42.641152 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:42.641158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:42.641164 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:42.641341 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:42.641925 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:42.641934 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:42.642265 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 09:40:42.642499 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 09:40:42.642828 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":108,"threads_started":1}
I20260812 09:40:42.646013 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 12519
I20260812 09:40:42.646210 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:42.646716 3660 external_mini_cluster.cc:1428] Setting key 9a34ccefbb8673d5e04557b4a26aa5c7
I20260812 09:40:42.648007 12671 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40855
I20260812 09:40:42.648170 12671 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:42.648619 12671 heartbeater.cc:511] Master 127.3.147.62:40855 requested a full tablet report, sending...
I20260812 09:40:42.650317 12429 ts_manager.cc:194] Registered new tserver with Master: 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963)
I20260812 09:40:42.651732 12429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:55447
I20260812 09:40:42.658236 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.649649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40855 (local address 127.3.147.1:41795)
0812 09:40:42.650023 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:42.650042 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:42.650157 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:42.650421 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:42.650427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:42.650698 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:42.650863 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:42.650875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.651965 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.651971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:42.653026 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:42.653036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.653596 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.653599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:42.653673 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.654161 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:42.654173 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:42.654512 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:42.656674 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:42.656677 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:42.656679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:42.656882 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:42.657200 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:42.657203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:42.657204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:42.657289 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:42.657770 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:42.657772 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:42.657851 (+ 79us) client_negotiation.cc:769] Sending connection context
0812 09:40:42.657942 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 09:40:42.658051 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260812 09:40:42.658344 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.649447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:41795 (local address 127.3.147.62:40855)
0812 09:40:42.649560 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:42.649564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:42.650141 (+ 577us) server_negotiation.cc:407] Connection header received
0812 09:40:42.650250 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:42.650253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:42.650295 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:42.650380 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:42.651004 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.651751 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.653166 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.653471 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.653575 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.654665 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:42.654671 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:42.654677 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:42.654709 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:42.656551 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.656988 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.656994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.656998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.657094 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.657389 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.657395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.657398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.657659 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:42.657779 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:42.658032 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 09:40:42.658153 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for oryx/127.3.147.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab.
Entry for principal oryx/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab.
I20260812 09:40:42.679761 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40855
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab
--principal=oryx/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:42.795099 12681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:42.795328 12681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:42.795372 12681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:42.802260 12681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:42.802342 12681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:42.802421 12681 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:42.802496 12681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:42.810850 12681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab
--principal=oryx/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:42.811862 12681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:42.812904 12681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:42.822463 12687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:42.822558 12690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:42.822543 12688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:42.822908 12681 server_base.cc:1034] running on GCE node
I20260812 09:40:42.823359 12681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:42.824018 12681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:42.825240 12681 hybrid_clock.cc:648] HybridClock initialized: now 1786527642825217 us; error 37 us; skew 500 ppm
Aug 12 09:40:42 dist-test-slave-cbbz krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527642, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:42.827980 12681 init.cc:377] Logged in from keytab as oryx/127.3.147.2@KRBTEST.COM (short username oryx)
I20260812 09:40:42.828900 12681 webserver.cc:460] Webserver started at http://127.3.147.2:44647/ using document root <none> and password file <none>
I20260812 09:40:42.829409 12681 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:42.829468 12681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:42.829638 12681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:42.831593 12681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "979027e14752410a820ad5c6f4bc14a5"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "60e8099e9558ca2f1fe76927b7bf0dd4"
server_key_iv: "bb35dafc158df9541679ad2bc35dbd08"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.832108 12681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "979027e14752410a820ad5c6f4bc14a5"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "60e8099e9558ca2f1fe76927b7bf0dd4"
server_key_iv: "bb35dafc158df9541679ad2bc35dbd08"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.835786 12681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:42.838264 12697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:42.839558 12681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:42.839663 12681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "979027e14752410a820ad5c6f4bc14a5"
format_stamp: "Formatted at 2026-08-12 09:40:42 on dist-test-slave-cbbz"
server_key: "60e8099e9558ca2f1fe76927b7bf0dd4"
server_key_iv: "bb35dafc158df9541679ad2bc35dbd08"
server_key_version: "encryptionkey@0"
I20260812 09:40:42.839748 12681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:42.852824 12681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:42.855710 12681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:42.855852 12681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:42.857303 12681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:42.857354 12681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:42.857405 12681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:42.857420 12681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:42 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527642, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:42.869356 12681 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:33993
I20260812 09:40:42.870446 12681 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:42.870543 12829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:33993 every 8 connection(s)
I20260812 09:40:42.872483 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.858209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:55625 (local address 127.3.147.62:40855)
0812 09:40:42.858351 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:42.858354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:42.859444 (+ 1090us) server_negotiation.cc:407] Connection header received
0812 09:40:42.860580 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:42.860584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:42.860689 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:42.860768 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:42.862193 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.863107 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.864524 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.864920 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.865052 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.868212 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:42.868216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:42.868219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:42.868249 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:42.869978 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.870675 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.870679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.870681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.870784 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.871186 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.871191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.871193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.871512 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:42.871619 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:42.872181 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 09:40:42.872307 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:40:42.873241 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.858714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40855 (local address 127.3.147.2:55625)
0812 09:40:42.859252 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:42.859297 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:42.860294 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:42.861189 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:42.861195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:42.861534 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:42.862008 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:42.862022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.863267 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.863270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:42.864410 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:42.864420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.865284 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.865290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:42.865492 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.866165 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:42.866182 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:42.867998 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:42.870105 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:42.870112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:42.870127 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:42.870527 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:42.870912 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:42.870916 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:42.870918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:42.871088 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:42.871623 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:42.871632 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:42.871900 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 09:40:42.872097 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 09:40:42.872432 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":158,"threads_started":1}
I20260812 09:40:42.874433 12830 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40855
I20260812 09:40:42.874594 12830 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:42.875214 12830 heartbeater.cc:511] Master 127.3.147.62:40855 requested a full tablet report, sending...
I20260812 09:40:42.876312 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 12681
I20260812 09:40:42.876443 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:42.876510 12430 ts_manager.cc:194] Registered new tserver with Master: 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993)
I20260812 09:40:42.876793 3660 external_mini_cluster.cc:1428] Setting key 4ac223b4bf72e00535cd430d9d9527fe
I20260812 09:40:42.877241 12430 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:55625
I20260812 09:40:42.885227 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.877288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:57731 (local address 127.3.147.62:40855)
0812 09:40:42.877391 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:42.877395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:42.877782 (+ 387us) server_negotiation.cc:407] Connection header received
0812 09:40:42.878102 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:42.878108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:42.878165 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:42.878251 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:42.878733 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.879640 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.880544 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.880778 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.880889 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.882038 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:42.882041 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:42.882044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:42.882067 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:42.883560 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.883990 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.883993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.883995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.884072 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:42.884389 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:42.884392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:42.884394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:42.884595 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:42.884714 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:42.884969 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 09:40:42.885075 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 09:40:42.885243 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:42.877167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40855 (local address 127.3.147.2:57731)
0812 09:40:42.877688 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:42.877701 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:42.877821 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:42.878271 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:42.878274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:42.878472 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:42.878585 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:42.878592 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.879795 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.879797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:42.880422 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:42.880431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:42.880894 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:42.880898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:42.880986 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:42.881582 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:42.881592 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:42.881904 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:42.883690 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:42.883693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:42.883695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:42.883889 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:42.884198 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:42.884200 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:42.884202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:42.884287 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:42.884706 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:42.884709 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:42.884806 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 09:40:42.884914 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 09:40:42.885051 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":148,"threads_started":1}
WARNING: no policy specified for oryx/127.3.147.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab.
Entry for principal oryx/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab.
I20260812 09:40:42.907724 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40855
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab
--principal=oryx/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:43.020390 12840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:43.020643 12840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:43.020712 12840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:43.024726 12840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:43.024793 12840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:43.024855 12840 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 09:40:43.024919 12840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:43.030138 12840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab
--principal=oryx/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.147.62:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:43.031133 12840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:43.032114 12840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:43.041425 12848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:43.041477 12846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:43.041453 12845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:43.041882 12840 server_base.cc:1034] running on GCE node
I20260812 09:40:43.042272 12840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:43.042920 12840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:43.044101 12840 hybrid_clock.cc:648] HybridClock initialized: now 1786527643044081 us; error 64 us; skew 500 ppm
Aug 12 09:40:43 dist-test-slave-cbbz krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527643, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:43.047111 12840 init.cc:377] Logged in from keytab as oryx/127.3.147.3@KRBTEST.COM (short username oryx)
I20260812 09:40:43.047951 12840 webserver.cc:460] Webserver started at http://127.3.147.3:46771/ using document root <none> and password file <none>
I20260812 09:40:43.048547 12840 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:43.048647 12840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:43.048874 12840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:43.051052 12840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "d58c84661a0548aab8e8ffe18677c971"
format_stamp: "Formatted at 2026-08-12 09:40:43 on dist-test-slave-cbbz"
server_key: "d5e826b80b471274b1dbe4c26fb1c744"
server_key_iv: "bfdb794dd2552c78ada7008cecddf611"
server_key_version: "encryptionkey@0"
I20260812 09:40:43.051728 12840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "d58c84661a0548aab8e8ffe18677c971"
format_stamp: "Formatted at 2026-08-12 09:40:43 on dist-test-slave-cbbz"
server_key: "d5e826b80b471274b1dbe4c26fb1c744"
server_key_iv: "bfdb794dd2552c78ada7008cecddf611"
server_key_version: "encryptionkey@0"
I20260812 09:40:43.055457 12840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 09:40:43.057976 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:43.059257 12840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 09:40:43.059412 12840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "d58c84661a0548aab8e8ffe18677c971"
format_stamp: "Formatted at 2026-08-12 09:40:43 on dist-test-slave-cbbz"
server_key: "d5e826b80b471274b1dbe4c26fb1c744"
server_key_iv: "bfdb794dd2552c78ada7008cecddf611"
server_key_version: "encryptionkey@0"
I20260812 09:40:43.059525 12840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:43.073218 12840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:43.076489 12840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:43.076690 12840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:43.078043 12840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:43.078117 12840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:43.078227 12840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:43.078276 12840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 09:40:43 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527643, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:43.089848 12840 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:45383
I20260812 09:40:43.089974 12989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:45383 every 8 connection(s)
I20260812 09:40:43.090821 12840 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:43.092866 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.079172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:39099 (local address 127.3.147.62:40855)
0812 09:40:43.079306 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.079310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.080372 (+ 1062us) server_negotiation.cc:407] Connection header received
0812 09:40:43.081501 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.081505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.081574 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.081687 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:43.083044 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.083578 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.084939 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.085166 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.085282 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.088921 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:43.088927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:43.088932 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:43.088961 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:43.090630 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:43.091240 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:43.091246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:43.091248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:43.091345 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:43.091791 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:43.091795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:43.091796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:43.091994 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:43.092075 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.092636 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.092738 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 09:40:43.093667 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.079664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40855 (local address 127.3.147.3:39099)
0812 09:40:43.080229 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.080271 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.081199 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.081901 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.081909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.082360 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:43.082877 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.082921 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.083716 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.083719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.084804 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.084814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.085323 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.085327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.085502 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.086068 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:43.086101 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:43.088734 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:43.090753 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:43.090760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:43.090776 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:43.091100 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:43.091460 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:43.091463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:43.091465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:43.091670 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:43.092100 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:43.092104 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:43.092420 (+ 316us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.092564 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.092814 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":156,"threads_started":1}
I20260812 09:40:43.094046 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 12840
I20260812 09:40:43.094205 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:43.094601 3660 external_mini_cluster.cc:1428] Setting key ffc20c92216d385e9bf1cee8459bed6e
I20260812 09:40:43.095013 12991 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40855
I20260812 09:40:43.095175 12991 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:43.095715 12991 heartbeater.cc:511] Master 127.3.147.62:40855 requested a full tablet report, sending...
I20260812 09:40:43.096776 12429 ts_manager.cc:194] Registered new tserver with Master: d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383)
I20260812 09:40:43.097546 12429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:39099
I20260812 09:40:43.104628 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.097353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40855 (local address 127.3.147.3:58885)
0812 09:40:43.097947 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.097962 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.098059 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.098333 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.098335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.098503 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:43.098610 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.098615 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.099598 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.099601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.100252 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.100262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.100708 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.100711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.100786 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.101209 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:43.101221 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:43.101521 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:43.103266 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:43.103268 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:43.103270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:43.103469 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:43.103755 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:43.103758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:43.103759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:43.103838 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:43.104203 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:43.104206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:43.104294 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.104393 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.104491 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":88,"threads_started":1}
I20260812 09:40:43.104637 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.097419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:58885 (local address 127.3.147.62:40855)
0812 09:40:43.097535 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.097538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.098029 (+ 491us) server_negotiation.cc:407] Connection header received
0812 09:40:43.098191 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.098194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.098239 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.098326 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:43.098727 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.099452 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.100361 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.100563 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.100645 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.101641 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:43.101644 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:43.101646 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:43.101676 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:43.103136 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:43.103577 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:43.103580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:43.103581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:43.103669 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:43.103918 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:43.103920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:43.103921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:43.104067 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:43.104223 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.104452 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.104539 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 09:40:43.108543 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 09:40:43 dist-test-slave-cbbz krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527643, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 09:40:43 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:43 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 09:40:43.133167 12488 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:43.127077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.3.147.62:40855)
0812 09:40:43.127237 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.127241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.127253 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:40:43.127292 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.127294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.127331 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.127401 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:43.128055 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.128635 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.129503 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.129722 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.129804 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.132973 (+ 3169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55502: BlockingRecv error: recv got EOF from 127.0.0.1:55502 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 09:40:43 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527643, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:43.142472 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.134566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.3.147.62:40855)
0812 09:40:43.134715 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.134720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.134837 (+ 117us) server_negotiation.cc:407] Connection header received
0812 09:40:43.134998 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.135001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.135045 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.135162 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:43.135749 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.136346 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.137141 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.137365 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.137475 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.139661 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:43.139664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:43.139666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:43.139688 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:43.141138 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:43.141514 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:43.141517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:43.141518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:43.141587 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:43.141816 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:43.141818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:43.141819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:43.141973 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:43.142073 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.142268 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.142357 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 09:40:43.145696 12429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:43.148913 12429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:43.163503 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.157445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53870 (local address 127.3.147.2:33993)
0812 09:40:43.157970 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.157987 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.158556 (+ 569us) server_negotiation.cc:407] Connection header received
0812 09:40:43.158655 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.158661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.158862 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.159029 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:43.159416 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.160342 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.161844 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.162690 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.162822 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.163006 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.163252 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.163350 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":138,"threads_started":1}
I20260812 09:40:43.163494 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.157116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:45383 (local address 127.0.0.1:59800)
0812 09:40:43.157480 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.157535 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.157669 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.158975 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.158983 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.159005 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:43.159152 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.159159 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.160592 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.160596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.161877 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.161887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.162894 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.162898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.163057 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.163104 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.163167 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.163338 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":4,"thread_start_us":61,"threads_started":1}
I20260812 09:40:43.163504 13020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.157125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:33993 (local address 127.0.0.1:53870)
0812 09:40:43.158330 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.158344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.158440 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.159167 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.159170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.159188 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:43.159299 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.159305 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.160449 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.160455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.161703 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.161713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.162826 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.162830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.163042 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.163088 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.163205 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.163346 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1141,"mutex_wait_us":985,"thread_start_us":118,"threads_started":1}
I20260812 09:40:43.164062 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.158825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48540 (local address 127.3.147.1:36963)
0812 09:40:43.159204 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.159230 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.159258 (+ 28us) server_negotiation.cc:407] Connection header received
0812 09:40:43.159330 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.159338 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.159515 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.159661 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:43.159894 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.160982 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.162322 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.162916 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.163027 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.163170 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.163822 (+ 652us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.163904 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":92,"threads_started":1}
I20260812 09:40:43.163744 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.158420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36963 (local address 127.0.0.1:48540)
0812 09:40:43.158662 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.158674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.158752 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.159664 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.159667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.159686 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:43.159767 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.159773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.161119 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.161134 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.162184 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.162191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.163323 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.163396 (+ 73us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.163482 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.163502 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.163584 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.163666 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
I20260812 09:40:43.165135 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.157876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.3.147.3:45383)
0812 09:40:43.158242 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.158254 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.158279 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:40:43.158363 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.158369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.158799 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.158975 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:43.159286 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.160454 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.162000 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.162759 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.164738 (+ 1979us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.164858 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.164896 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.164964 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":137,"threads_started":1}
I20260812 09:40:43.167212 12916 tablet_service.cc:1467] Processing CreateTablet for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 (DEFAULT_TABLE table=test-table [id=e4e5da59af524e478452ff3c806ad3bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:43.167133 12749 tablet_service.cc:1467] Processing CreateTablet for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 (DEFAULT_TABLE table=test-table [id=e4e5da59af524e478452ff3c806ad3bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:43.167408 12593 tablet_service.cc:1467] Processing CreateTablet for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 (DEFAULT_TABLE table=test-table [id=e4e5da59af524e478452ff3c806ad3bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:43.168282 12749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.168280 12593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.168282 12916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.175204 13023 tablet_bootstrap.cc:492] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Bootstrap starting.
I20260812 09:40:43.176544 13026 tablet_bootstrap.cc:492] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Bootstrap starting.
I20260812 09:40:43.176592 13025 tablet_bootstrap.cc:492] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Bootstrap starting.
I20260812 09:40:43.177688 13023 tablet_bootstrap.cc:654] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.178663 13026 tablet_bootstrap.cc:654] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.178663 13025 tablet_bootstrap.cc:654] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.178633 13023 log.cc:824] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:43.179512 13025 log.cc:824] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:43.179512 13026 log.cc:824] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:43.180639 13023 tablet_bootstrap.cc:492] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: No bootstrap required, opened a new log
I20260812 09:40:43.180851 13023 ts_tablet_manager.cc:1397] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260812 09:40:43.181391 13026 tablet_bootstrap.cc:492] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: No bootstrap required, opened a new log
I20260812 09:40:43.181386 13025 tablet_bootstrap.cc:492] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: No bootstrap required, opened a new log
I20260812 09:40:43.181569 13026 ts_tablet_manager.cc:1397] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:40:43.181569 13025 ts_tablet_manager.cc:1397] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 09:40:43.183739 13023 raft_consensus.cc:348] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.183969 13023 raft_consensus.cc:374] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.184022 13023 raft_consensus.cc:729] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58c84661a0548aab8e8ffe18677c971, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.184481 13023 consensus_queue.cc:260] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.185515 13023 ts_tablet_manager.cc:1428] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260812 09:40:43.185554 13026 raft_consensus.cc:348] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.185799 13026 raft_consensus.cc:374] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.185868 13026 raft_consensus.cc:729] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979027e14752410a820ad5c6f4bc14a5, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.185877 12991 heartbeater.cc:503] Master 127.3.147.62:40855 was elected leader, sending a full tablet report...
I20260812 09:40:43.185899 13025 raft_consensus.cc:348] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.186124 13025 raft_consensus.cc:374] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.186178 13025 raft_consensus.cc:729] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d9aa44630d479aa34ef0db5965ccb8, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.186283 13026 consensus_queue.cc:260] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.186544 13025 consensus_queue.cc:260] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.187211 13026 ts_tablet_manager.cc:1428] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:40:43.187263 12830 heartbeater.cc:503] Master 127.3.147.62:40855 was elected leader, sending a full tablet report...
I20260812 09:40:43.187331 13025 ts_tablet_manager.cc:1428] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 09:40:43.188179 12671 heartbeater.cc:503] Master 127.3.147.62:40855 was elected leader, sending a full tablet report...
W20260812 09:40:43.341858 12992 tablet.cc:2369] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:40:43.371793 12832 tablet.cc:2369] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:40:43.392956 12672 tablet.cc:2369] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:40:43.566605 13031 raft_consensus.cc:482] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:43.566854 13031 raft_consensus.cc:504] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.568312 13031 leader_election.cc:288] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993), 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963)
I20260812 09:40:43.568416 13033 raft_consensus.cc:482] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:43.568626 13033 raft_consensus.cc:504] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.570406 13033 leader_election.cc:288] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383), 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993)
I20260812 09:40:43.573815 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.568630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:33993 (local address 127.3.147.3:41331)
0812 09:40:43.568818 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.568837 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.568959 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.569493 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.569497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.569521 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:43.569685 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.569694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.570925 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.570931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.572295 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.572308 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.573342 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.573349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.573468 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.573496 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.573583 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.573662 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 09:40:43.574311 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.568692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:41331 (local address 127.3.147.2:33993)
0812 09:40:43.568836 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.568841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.568974 (+ 133us) server_negotiation.cc:407] Connection header received
0812 09:40:43.569154 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.569158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.569219 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.569305 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:43.569828 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.570709 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.572469 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.573091 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.573202 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.573275 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.574098 (+ 823us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.574180 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 09:40:43.574898 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce82ebf6a8842e3a922a39cbcb6d7a3" candidate_uuid: "d58c84661a0548aab8e8ffe18677c971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5" is_pre_election: true
I20260812 09:40:43.575047 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.569102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:45387 (local address 127.3.147.1:36963)
0812 09:40:43.569262 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.569270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.569487 (+ 217us) server_negotiation.cc:407] Connection header received
0812 09:40:43.569626 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.569630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.569687 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.569791 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:43.571025 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.571722 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.573219 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.573970 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.574056 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.574110 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.574849 (+ 739us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.574919 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":52}
I20260812 09:40:43.575197 12770 raft_consensus.cc:2454] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58c84661a0548aab8e8ffe18677c971 in term 0.
I20260812 09:40:43.575585 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.570377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:34493 (local address 127.3.147.2:33993)
0812 09:40:43.571100 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.571104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.571116 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:40:43.571169 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.571172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.571215 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.571294 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:43.571646 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.572405 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.573971 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.574655 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.574761 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.574834 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.575379 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.575460 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":152,"threads_started":1}
I20260812 09:40:43.575723 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.570082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:33993 (local address 127.3.147.1:34493)
0812 09:40:43.570634 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.570651 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.570758 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.571404 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.571407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.571424 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:43.571533 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.571539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.572531 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.572534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.573830 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.573841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.574887 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.574890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.574980 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.575000 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.575068 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.575135 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":49,"thread_start_us":94,"threads_started":1}
I20260812 09:40:43.575737 12860 leader_election.cc:302] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 979027e14752410a820ad5c6f4bc14a5, d58c84661a0548aab8e8ffe18677c971; no voters:
I20260812 09:40:43.575937 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.570124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:57637 (local address 127.3.147.3:45383)
0812 09:40:43.570354 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.570378 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.570999 (+ 621us) server_negotiation.cc:407] Connection header received
0812 09:40:43.571212 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.571215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.571260 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.571313 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:40:43.571754 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.572928 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.574549 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.575143 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.575254 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.575338 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.575746 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.575818 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":152}
I20260812 09:40:43.576026 13031 raft_consensus.cc:2781] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:43.576129 13031 raft_consensus.cc:482] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:43.576190 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.568940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36963 (local address 127.3.147.3:45387)
0812 09:40:43.569364 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.569380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.569508 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.570686 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.570689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.570708 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:43.570882 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.570894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.571870 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.571873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.573082 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.573093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.574519 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.574525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.574624 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.574733 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.575452 (+ 719us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.575560 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":44,"thread_start_us":81,"threads_started":1}
I20260812 09:40:43.576218 13031 raft_consensus.cc:3037] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.576300 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce82ebf6a8842e3a922a39cbcb6d7a3" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5" is_pre_election: true
I20260812 09:40:43.576373 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.570122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:45383 (local address 127.3.147.1:57637)
0812 09:40:43.570918 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.570932 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.571091 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.571456 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.571459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.571479 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:40:43.571617 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.571625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.573089 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.573093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.574384 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.574398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.575442 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.575447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.575585 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.575613 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.576113 (+ 500us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.576216 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260812 09:40:43.576445 12770 raft_consensus.cc:2454] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 0.
I20260812 09:40:43.576850 12538 leader_election.cc:302] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, 979027e14752410a820ad5c6f4bc14a5; no voters:
I20260812 09:40:43.576972 12937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce82ebf6a8842e3a922a39cbcb6d7a3" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58c84661a0548aab8e8ffe18677c971" is_pre_election: true
I20260812 09:40:43.577049 12617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce82ebf6a8842e3a922a39cbcb6d7a3" candidate_uuid: "d58c84661a0548aab8e8ffe18677c971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d9aa44630d479aa34ef0db5965ccb8" is_pre_election: true
I20260812 09:40:43.577149 13033 raft_consensus.cc:2781] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:43.577247 12617 raft_consensus.cc:2454] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58c84661a0548aab8e8ffe18677c971 in term 0.
I20260812 09:40:43.577306 13033 raft_consensus.cc:482] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:43.577359 13033 raft_consensus.cc:3037] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.577713 13031 raft_consensus.cc:504] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.578012 12937 raft_consensus.cc:2379] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35d9aa44630d479aa34ef0db5965ccb8 in current term 1: Already voted for candidate d58c84661a0548aab8e8ffe18677c971 in this term.
I20260812 09:40:43.578028 13031 leader_election.cc:288] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [CANDIDATE]: Term 1 election: Requested vote from peers 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993), 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963)
I20260812 09:40:43.578745 13033 raft_consensus.cc:504] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.578804 12617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce82ebf6a8842e3a922a39cbcb6d7a3" candidate_uuid: "d58c84661a0548aab8e8ffe18677c971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d9aa44630d479aa34ef0db5965ccb8"
I20260812 09:40:43.578980 12617 raft_consensus.cc:2379] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d58c84661a0548aab8e8ffe18677c971 in current term 1: Already voted for candidate 35d9aa44630d479aa34ef0db5965ccb8 in this term.
I20260812 09:40:43.579016 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce82ebf6a8842e3a922a39cbcb6d7a3" candidate_uuid: "d58c84661a0548aab8e8ffe18677c971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5"
I20260812 09:40:43.579167 12770 raft_consensus.cc:3037] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.580320 12770 raft_consensus.cc:2454] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58c84661a0548aab8e8ffe18677c971 in term 1.
I20260812 09:40:43.580422 13033 leader_election.cc:288] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 election: Requested vote from peers d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383), 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993)
I20260812 09:40:43.580653 12860 leader_election.cc:302] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 979027e14752410a820ad5c6f4bc14a5, d58c84661a0548aab8e8ffe18677c971; no voters: 35d9aa44630d479aa34ef0db5965ccb8
I20260812 09:40:43.580828 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce82ebf6a8842e3a922a39cbcb6d7a3" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5"
I20260812 09:40:43.580948 12937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ce82ebf6a8842e3a922a39cbcb6d7a3" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58c84661a0548aab8e8ffe18677c971"
I20260812 09:40:43.581012 12770 raft_consensus.cc:2379] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 35d9aa44630d479aa34ef0db5965ccb8 in current term 1: Already voted for candidate d58c84661a0548aab8e8ffe18677c971 in this term.
I20260812 09:40:43.581079 12937 raft_consensus.cc:2379] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 35d9aa44630d479aa34ef0db5965ccb8 in current term 1: Already voted for candidate d58c84661a0548aab8e8ffe18677c971 in this term.
I20260812 09:40:43.581265 13031 raft_consensus.cc:2781] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:43.581459 13031 raft_consensus.cc:686] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 1 LEADER]: Becoming Leader. State: Replica: d58c84661a0548aab8e8ffe18677c971, State: Running, Role: LEADER
I20260812 09:40:43.581480 12538 leader_election.cc:302] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8; no voters: 979027e14752410a820ad5c6f4bc14a5, d58c84661a0548aab8e8ffe18677c971
I20260812 09:40:43.581686 13033 raft_consensus.cc:2726] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 09:40:43.581785 13031 consensus_queue.cc:237] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } }
I20260812 09:40:43.584903 12429 catalog_manager.cc:5495] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 reported cstate change: term changed from 0 to 1, leader changed from <none> to d58c84661a0548aab8e8ffe18677c971 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "d58c84661a0548aab8e8ffe18677c971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } health_report { overall_health: UNKNOWN } } }
I20260812 09:40:43.610577 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.606951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.3.147.3:45383)
0812 09:40:43.607092 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.607097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.607113 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:40:43.607311 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.607314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.607368 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.607488 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:43.607729 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.608300 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.609162 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.609392 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.609500 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.609776 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:43.610168 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:43.610296 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.610359 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.610412 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 09:40:43.616117 12770 raft_consensus.cc:1261] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Refusing update from remote peer d58c84661a0548aab8e8ffe18677c971: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:43.616199 12617 raft_consensus.cc:1261] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Refusing update from remote peer d58c84661a0548aab8e8ffe18677c971: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:43.616768 13040 consensus_queue.cc:1035] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:43.616966 13031 consensus_queue.cc:1035] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:43.623215 13044 mvcc.cc:205] Tried to move back new op lower bound from 7317617228244963328 to 7317617228113895424. Current Snapshot: MvccSnapshot[applied={T|T < 7317617228244963328}]
I20260812 09:40:43.631897 12429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:43.632061 12429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:43.634465 12429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: Sending DeleteTablet for 3 replicas of tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3
I20260812 09:40:43.635340 12916 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:43 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:59800
I20260812 09:40:43.635627 12593 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:43 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:48540
I20260812 09:40:43.635812 12749 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:43 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:53870
I20260812 09:40:43.636001 12429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 09:40:43.636301 13053 tablet_replica.cc:330] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: stopping tablet replica
W20260812 09:40:43.636418 12429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:43.636472 13055 tablet_replica.cc:330] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: stopping tablet replica
I20260812 09:40:43.636612 13053 raft_consensus.cc:2229] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:43.636917 13053 raft_consensus.cc:2258] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:43.636934 13055 raft_consensus.cc:2229] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:43.637312 13055 raft_consensus.cc:2258] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:43.637835 13054 tablet_replica.cc:330] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: stopping tablet replica
I20260812 09:40:43.638104 13054 raft_consensus.cc:2229] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:43.638867 13055 ts_tablet_manager.cc:1905] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:43.639297 13053 ts_tablet_manager.cc:1905] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:43.641139 13054 raft_consensus.cc:2258] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:43.642083 13055 ts_tablet_manager.cc:1918] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:43.642266 13055 log.cc:1196] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/1ce82ebf6a8842e3a922a39cbcb6d7a3
I20260812 09:40:43.642681 13055 ts_tablet_manager.cc:1939] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 979027e14752410a820ad5c6f4bc14a5: Deleting consensus metadata
I20260812 09:40:43.643168 13054 ts_tablet_manager.cc:1905] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:43.643616 12593 tablet_service.cc:1467] Processing CreateTablet for tablet 499520e039a94bc79cbff0dd4aa451a6 (DEFAULT_TABLE table=test-table [id=eab47c9f8304412a948ab12c17810026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:43.643741 12916 tablet_service.cc:1467] Processing CreateTablet for tablet 499520e039a94bc79cbff0dd4aa451a6 (DEFAULT_TABLE table=test-table [id=eab47c9f8304412a948ab12c17810026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:43.643893 12593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499520e039a94bc79cbff0dd4aa451a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.643996 12916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499520e039a94bc79cbff0dd4aa451a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.645184 12749 tablet_service.cc:1467] Processing CreateTablet for tablet 499520e039a94bc79cbff0dd4aa451a6 (DEFAULT_TABLE table=test-table [id=eab47c9f8304412a948ab12c17810026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:43.645406 12749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 499520e039a94bc79cbff0dd4aa451a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.646450 13025 tablet_bootstrap.cc:492] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: Bootstrap starting.
I20260812 09:40:43.646739 13054 ts_tablet_manager.cc:1918] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:43.646819 13054 log.cc:1196] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/1ce82ebf6a8842e3a922a39cbcb6d7a3
I20260812 09:40:43.647126 13054 ts_tablet_manager.cc:1939] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P 35d9aa44630d479aa34ef0db5965ccb8: Deleting consensus metadata
I20260812 09:40:43.647509 13025 tablet_bootstrap.cc:654] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.647615 13026 tablet_bootstrap.cc:492] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: Bootstrap starting.
I20260812 09:40:43.649008 13026 tablet_bootstrap.cc:654] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.649745 13025 tablet_bootstrap.cc:492] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: No bootstrap required, opened a new log
I20260812 09:40:43.649830 13025 ts_tablet_manager.cc:1397] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:43.650385 13025 raft_consensus.cc:348] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.650487 13025 raft_consensus.cc:374] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.650520 13025 raft_consensus.cc:729] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d9aa44630d479aa34ef0db5965ccb8, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.650631 13025 consensus_queue.cc:260] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.650882 13025 ts_tablet_manager.cc:1428] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:43.652477 12417 catalog_manager.cc:4841] TS 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963): tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 (table test-table [id=e4e5da59af524e478452ff3c806ad3bd]) successfully deleted
I20260812 09:40:43.652808 12417 catalog_manager.cc:4841] TS 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993): tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 (table test-table [id=e4e5da59af524e478452ff3c806ad3bd]) successfully deleted
I20260812 09:40:43.655113 13023 tablet_bootstrap.cc:492] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: Bootstrap starting.
I20260812 09:40:43.656076 13023 tablet_bootstrap.cc:654] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.657159 13053 ts_tablet_manager.cc:1918] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:43.657258 13053 log.cc:1196] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/1ce82ebf6a8842e3a922a39cbcb6d7a3
I20260812 09:40:43.657541 13053 ts_tablet_manager.cc:1939] T 1ce82ebf6a8842e3a922a39cbcb6d7a3 P d58c84661a0548aab8e8ffe18677c971: Deleting consensus metadata
I20260812 09:40:43.658317 13023 tablet_bootstrap.cc:492] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: No bootstrap required, opened a new log
I20260812 09:40:43.658396 13023 ts_tablet_manager.cc:1397] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 09:40:43.658794 13023 raft_consensus.cc:348] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.658835 12415 catalog_manager.cc:4841] TS d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383): tablet 1ce82ebf6a8842e3a922a39cbcb6d7a3 (table test-table [id=e4e5da59af524e478452ff3c806ad3bd]) successfully deleted
I20260812 09:40:43.658900 13023 raft_consensus.cc:374] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.658933 13023 raft_consensus.cc:729] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58c84661a0548aab8e8ffe18677c971, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.659072 13023 consensus_queue.cc:260] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.659312 13023 ts_tablet_manager.cc:1428] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 09:40:43.660190 13026 tablet_bootstrap.cc:492] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: No bootstrap required, opened a new log
I20260812 09:40:43.660274 13026 ts_tablet_manager.cc:1397] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260812 09:40:43.660717 13026 raft_consensus.cc:348] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.660823 13026 raft_consensus.cc:374] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.660851 13026 raft_consensus.cc:729] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979027e14752410a820ad5c6f4bc14a5, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.660951 13026 consensus_queue.cc:260] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.661168 13026 ts_tablet_manager.cc:1428] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:43.744243 13033 raft_consensus.cc:482] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:43.744395 13033 raft_consensus.cc:504] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.744859 13033 leader_election.cc:288] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993), d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383)
I20260812 09:40:43.745558 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499520e039a94bc79cbff0dd4aa451a6" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5" is_pre_election: true
I20260812 09:40:43.745558 12937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499520e039a94bc79cbff0dd4aa451a6" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58c84661a0548aab8e8ffe18677c971" is_pre_election: true
I20260812 09:40:43.745724 12770 raft_consensus.cc:2454] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 0.
I20260812 09:40:43.745760 12937 raft_consensus.cc:2454] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 0.
I20260812 09:40:43.746109 12538 leader_election.cc:302] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, 979027e14752410a820ad5c6f4bc14a5; no voters:
I20260812 09:40:43.746410 13033 raft_consensus.cc:2781] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:43.746469 13033 raft_consensus.cc:482] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:43.746505 13033 raft_consensus.cc:3037] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.747848 13033 raft_consensus.cc:504] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.748270 13033 leader_election.cc:288] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 election: Requested vote from peers 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993), d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383)
I20260812 09:40:43.748587 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499520e039a94bc79cbff0dd4aa451a6" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5"
I20260812 09:40:43.748780 12770 raft_consensus.cc:3037] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.748842 12937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "499520e039a94bc79cbff0dd4aa451a6" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58c84661a0548aab8e8ffe18677c971"
I20260812 09:40:43.748960 12937 raft_consensus.cc:3037] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.750010 12770 raft_consensus.cc:2454] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 1.
I20260812 09:40:43.750203 12937 raft_consensus.cc:2454] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 1.
I20260812 09:40:43.750372 12538 leader_election.cc:302] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, 979027e14752410a820ad5c6f4bc14a5; no voters:
I20260812 09:40:43.750535 13033 raft_consensus.cc:2781] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:43.750762 13033 raft_consensus.cc:686] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 LEADER]: Becoming Leader. State: Replica: 35d9aa44630d479aa34ef0db5965ccb8, State: Running, Role: LEADER
I20260812 09:40:43.751020 13033 consensus_queue.cc:237] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.753515 12430 catalog_manager.cc:5495] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1). New cstate: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } health_report { overall_health: UNKNOWN } } }
Aug 12 09:40:43 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527642, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.62@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:43.802440 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.793831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40855 (local address 127.0.0.1:55510)
0812 09:40:43.794179 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.794196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.794309 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.794614 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.794617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.795080 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:43.795216 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.795223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.796119 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.796123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.796801 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.796810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.797259 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.797261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.797322 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.797854 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:43.797865 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:43.799297 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:43.801044 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:43.801053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:43.801058 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:43.801282 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:43.801522 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:43.801524 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:43.801526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:43.801604 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:43.801944 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:43.801949 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:43.802041 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.802149 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.802253 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":118,"threads_started":1}
I20260812 09:40:43.802443 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.793865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.3.147.62:40855)
0812 09:40:43.794228 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.794236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.794253 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:40:43.794403 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.794405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.794463 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.794566 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:43.795359 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.795995 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.796907 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.797163 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.797242 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.799429 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:43.799432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:43.799434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:43.799455 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:43.800949 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:43.801367 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:43.801370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:43.801371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:43.801444 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:43.801682 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:43.801684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:43.801685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:43.801854 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:43.801928 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.802210 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.802302 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":100,"threads_started":1}
I20260812 09:40:43.806197 12430 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:55510:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 09:40:43.806820 12430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:43.811668 12593 tablet_service.cc:1467] Processing CreateTablet for tablet 06398ef019a64467bfa9faef45cf38eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=054b6f4817f2477bac5098630b6479e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:43.811800 12749 tablet_service.cc:1467] Processing CreateTablet for tablet 06398ef019a64467bfa9faef45cf38eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=054b6f4817f2477bac5098630b6479e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:43.811896 12916 tablet_service.cc:1467] Processing CreateTablet for tablet 06398ef019a64467bfa9faef45cf38eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=054b6f4817f2477bac5098630b6479e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 09:40:43.812119 12593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06398ef019a64467bfa9faef45cf38eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.812295 12749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06398ef019a64467bfa9faef45cf38eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.812290 12916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06398ef019a64467bfa9faef45cf38eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:43.814754 13025 tablet_bootstrap.cc:492] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: Bootstrap starting.
I20260812 09:40:43.814996 13023 tablet_bootstrap.cc:492] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: Bootstrap starting.
I20260812 09:40:43.815584 13026 tablet_bootstrap.cc:492] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: Bootstrap starting.
I20260812 09:40:43.815834 13025 tablet_bootstrap.cc:654] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.815874 13023 tablet_bootstrap.cc:654] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.816769 13026 tablet_bootstrap.cc:654] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:43.817131 13025 tablet_bootstrap.cc:492] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: No bootstrap required, opened a new log
I20260812 09:40:43.817160 13023 tablet_bootstrap.cc:492] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: No bootstrap required, opened a new log
I20260812 09:40:43.817200 13025 ts_tablet_manager.cc:1397] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:43.817239 13023 ts_tablet_manager.cc:1397] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 09:40:43.817653 13025 raft_consensus.cc:348] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.817674 13023 raft_consensus.cc:348] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.817777 13025 raft_consensus.cc:374] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.817790 13023 raft_consensus.cc:374] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.817817 13025 raft_consensus.cc:729] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d9aa44630d479aa34ef0db5965ccb8, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.817826 13023 raft_consensus.cc:729] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58c84661a0548aab8e8ffe18677c971, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.817962 13025 consensus_queue.cc:260] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.817976 13023 consensus_queue.cc:260] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.818267 13025 ts_tablet_manager.cc:1428] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 09:40:43.818202 13033 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.818284 13023 ts_tablet_manager.cc:1428] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 09:40:43.818352 13033 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:43.818411 13050 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.818526 13050 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:43.819001 13026 tablet_bootstrap.cc:492] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: No bootstrap required, opened a new log
I20260812 09:40:43.819092 13026 ts_tablet_manager.cc:1397] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 09:40:43.819530 13026 raft_consensus.cc:348] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.819643 13026 raft_consensus.cc:374] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:43.819687 13026 raft_consensus.cc:729] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979027e14752410a820ad5c6f4bc14a5, State: Initialized, Role: FOLLOWER
I20260812 09:40:43.819825 13026 consensus_queue.cc:260] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.820101 13026 ts_tablet_manager.cc:1428] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 09:40:43.820124 13032 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.820294 13032 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:43.950346 13033 raft_consensus.cc:482] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:43.950557 13033 raft_consensus.cc:504] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.951049 13033 leader_election.cc:288] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993), d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383)
I20260812 09:40:43.951586 12937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06398ef019a64467bfa9faef45cf38eb" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58c84661a0548aab8e8ffe18677c971" is_pre_election: true
I20260812 09:40:43.951593 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06398ef019a64467bfa9faef45cf38eb" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5" is_pre_election: true
I20260812 09:40:43.951737 12937 raft_consensus.cc:2454] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 0.
I20260812 09:40:43.951774 12770 raft_consensus.cc:2454] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 0.
I20260812 09:40:43.952247 12536 leader_election.cc:302] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, 979027e14752410a820ad5c6f4bc14a5, d58c84661a0548aab8e8ffe18677c971; no voters:
I20260812 09:40:43.952428 13033 raft_consensus.cc:2781] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:43.952495 13033 raft_consensus.cc:482] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:43.952569 13033 raft_consensus.cc:3037] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.953569 13033 raft_consensus.cc:504] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.953943 13033 leader_election.cc:288] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 election: Requested vote from peers 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993), d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383)
I20260812 09:40:43.954301 12937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06398ef019a64467bfa9faef45cf38eb" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58c84661a0548aab8e8ffe18677c971"
I20260812 09:40:43.954313 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06398ef019a64467bfa9faef45cf38eb" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5"
I20260812 09:40:43.954406 12937 raft_consensus.cc:3037] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.954452 12770 raft_consensus.cc:3037] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:43.955389 12937 raft_consensus.cc:2454] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 1.
I20260812 09:40:43.955549 12770 raft_consensus.cc:2454] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 1.
I20260812 09:40:43.955724 12536 leader_election.cc:302] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, d58c84661a0548aab8e8ffe18677c971; no voters:
I20260812 09:40:43.955899 13033 raft_consensus.cc:2781] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:43.955988 13033 raft_consensus.cc:686] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 LEADER]: Becoming Leader. State: Replica: 35d9aa44630d479aa34ef0db5965ccb8, State: Running, Role: LEADER
I20260812 09:40:43.956171 13033 consensus_queue.cc:237] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:43.957268 13059 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: TxnStatusTablet state changed. Reason: New leader 35d9aa44630d479aa34ef0db5965ccb8. Latest consensus state: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.957423 13059 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:43.957762 13072 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 09:40:43.957940 13072 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 09:40:43.958007 12428 catalog_manager.cc:5495] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1). New cstate: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } health_report { overall_health: UNKNOWN } } }
I20260812 09:40:43.971087 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.967332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.3.147.1:36963)
0812 09:40:43.967465 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.967470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.967483 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:43.967647 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.967651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.967712 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.967806 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:43.968058 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.968655 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.969514 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.969797 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.969905 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.970211 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:43.970675 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:43.970774 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.970858 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.970910 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 09:40:43.971064 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.967208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36963 (local address 127.0.0.1:48546)
0812 09:40:43.967354 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.967374 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.967481 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.967832 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.967835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.967844 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 09:40:43.967952 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.967957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.968764 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.968770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.969389 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.969398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.969923 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.969927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.970019 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.970060 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 09:40:43.970762 (+ 702us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 09:40:43.970779 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.970846 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.970934 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 09:40:43.973994 12770 raft_consensus.cc:1261] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Refusing update from remote peer 35d9aa44630d479aa34ef0db5965ccb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:43.974028 12937 raft_consensus.cc:1261] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Refusing update from remote peer 35d9aa44630d479aa34ef0db5965ccb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:43.974488 13033 consensus_queue.cc:1035] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:43.974678 13059 consensus_queue.cc:1035] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:43.976403 13050 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: TxnStatusTablet state changed. Reason: New leader 35d9aa44630d479aa34ef0db5965ccb8. Latest consensus state: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.976433 13032 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: TxnStatusTablet state changed. Reason: New leader 35d9aa44630d479aa34ef0db5965ccb8. Latest consensus state: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.976531 13050 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:43.976540 13032 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:43.977897 13033 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.978000 13033 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:43.979198 13032 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.979300 13032 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P 979027e14752410a820ad5c6f4bc14a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:43.985702 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.982386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.3.147.1:36963)
0812 09:40:43.982509 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.982513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.982562 (+ 49us) server_negotiation.cc:407] Connection header received
0812 09:40:43.982673 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.982676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.982721 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.982800 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:43.983005 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.983777 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.984616 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.984960 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.985065 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.985289 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:43.985367 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:43.985440 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.985512 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.985561 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:40:43.987020 13059 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.987126 13059 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P 35d9aa44630d479aa34ef0db5965ccb8: This TxnStatusTablet replica's current role is: LEADER
I20260812 09:40:43.988011 13050 tablet_replica.cc:439] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } } }
I20260812 09:40:43.988124 13050 tablet_replica.cc:442] T 06398ef019a64467bfa9faef45cf38eb P d58c84661a0548aab8e8ffe18677c971: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 09:40:43.997087 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.993355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:58905 (local address 127.3.147.1:36963)
0812 09:40:43.993464 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:43.993468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:43.993668 (+ 200us) server_negotiation.cc:407] Connection header received
0812 09:40:43.993745 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:43.993748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:43.993793 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:43.993870 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:43.994180 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.994958 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.995810 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.996158 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.996280 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.996580 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:43.996662 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:43.996740 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:43.996903 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 09:40:43.996952 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":32}
I20260812 09:40:43.997098 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:43.993247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36963 (local address 127.3.147.1:58905)
0812 09:40:43.993579 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:43.993598 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:43.993680 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:43.993905 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:43.993910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:43.993921 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 09:40:43.994060 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.994066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.995063 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.995065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.995701 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:43.995709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:43.996269 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:43.996271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:43.996410 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:43.996452 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 09:40:43.996781 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 09:40:43.996789 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 09:40:43.996877 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 09:40:43.996947 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":17,"thread_start_us":110,"threads_started":1}
I20260812 09:40:44.003935 12937 raft_consensus.cc:1261] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Refusing update from remote peer 35d9aa44630d479aa34ef0db5965ccb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:44.003968 12770 raft_consensus.cc:1261] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Refusing update from remote peer 35d9aa44630d479aa34ef0db5965ccb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:44.004392 13033 consensus_queue.cc:1035] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:44.004572 13059 consensus_queue.cc:1035] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:44.008534 13045 mvcc.cc:205] Tried to move back new op lower bound from 7317617229837045760 to 7317617228806893568. Current Snapshot: MvccSnapshot[applied={T|T < 7317617229837045760}]
I20260812 09:40:44.047669 12430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:44.047863 12430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:44.049919 12430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: Sending DeleteTablet for 3 replicas of tablet 499520e039a94bc79cbff0dd4aa451a6
I20260812 09:40:44.050514 12749 tablet_service.cc:1514] Processing DeleteTablet for tablet 499520e039a94bc79cbff0dd4aa451a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:53870
I20260812 09:40:44.050514 12916 tablet_service.cc:1514] Processing DeleteTablet for tablet 499520e039a94bc79cbff0dd4aa451a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:59800
I20260812 09:40:44.050704 13055 tablet_replica.cc:330] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: stopping tablet replica
I20260812 09:40:44.050825 13055 raft_consensus.cc:2229] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:44.050969 13055 raft_consensus.cc:2258] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.051187 13053 tablet_replica.cc:330] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: stopping tablet replica
I20260812 09:40:44.051337 13053 raft_consensus.cc:2229] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:44.051456 13053 raft_consensus.cc:2258] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.051874 13055 ts_tablet_manager.cc:1905] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:44.054843 13053 ts_tablet_manager.cc:1905] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:44.056936 13055 ts_tablet_manager.cc:1918] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:44.057107 13055 log.cc:1196] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/499520e039a94bc79cbff0dd4aa451a6
I20260812 09:40:44.057462 13055 ts_tablet_manager.cc:1939] T 499520e039a94bc79cbff0dd4aa451a6 P 979027e14752410a820ad5c6f4bc14a5: Deleting consensus metadata
I20260812 09:40:44.058369 12417 catalog_manager.cc:4841] TS 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993): tablet 499520e039a94bc79cbff0dd4aa451a6 (table test-table [id=eab47c9f8304412a948ab12c17810026]) successfully deleted
I20260812 09:40:44.058905 13053 ts_tablet_manager.cc:1918] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:44.058980 13053 log.cc:1196] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/499520e039a94bc79cbff0dd4aa451a6
I20260812 09:40:44.059278 13053 ts_tablet_manager.cc:1939] T 499520e039a94bc79cbff0dd4aa451a6 P d58c84661a0548aab8e8ffe18677c971: Deleting consensus metadata
I20260812 09:40:44.059495 12593 tablet_service.cc:1514] Processing DeleteTablet for tablet 499520e039a94bc79cbff0dd4aa451a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:48540
I20260812 09:40:44.059706 13054 tablet_replica.cc:330] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: stopping tablet replica
I20260812 09:40:44.059842 13054 raft_consensus.cc:2229] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:44.060015 12415 catalog_manager.cc:4841] TS d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383): tablet 499520e039a94bc79cbff0dd4aa451a6 (table test-table [id=eab47c9f8304412a948ab12c17810026]) successfully deleted
I20260812 09:40:44.060189 13054 raft_consensus.cc:2258] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.060952 13054 ts_tablet_manager.cc:1905] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:44.063838 13054 ts_tablet_manager.cc:1918] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:44.063925 13054 log.cc:1196] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/499520e039a94bc79cbff0dd4aa451a6
I20260812 09:40:44.064234 13054 ts_tablet_manager.cc:1939] T 499520e039a94bc79cbff0dd4aa451a6 P 35d9aa44630d479aa34ef0db5965ccb8: Deleting consensus metadata
I20260812 09:40:44.064976 12417 catalog_manager.cc:4841] TS 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963): tablet 499520e039a94bc79cbff0dd4aa451a6 (table test-table [id=eab47c9f8304412a948ab12c17810026]) successfully deleted
Aug 12 09:40:44 dist-test-slave-cbbz krb5kdc[12381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 09:40:44 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:44 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 09:40:44.084619 13067 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:44.077978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55522 (local address 127.3.147.62:40855)
0812 09:40:44.078128 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:44.078133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:44.078288 (+ 155us) server_negotiation.cc:407] Connection header received
0812 09:40:44.078403 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:44.078406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:44.078461 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:44.078542 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:44.079236 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:44.080022 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:44.080923 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:44.081277 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:44.081392 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:44.084496 (+ 3104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55522: BlockingRecv error: recv got EOF from 127.0.0.1:55522 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 09:40:44 dist-test-slave-cbbz krb5kdc[12381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM
I20260812 09:40:44.096066 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:44.085980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.3.147.62:40855)
0812 09:40:44.086084 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:44.086089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:44.086446 (+ 357us) server_negotiation.cc:407] Connection header received
0812 09:40:44.086547 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:44.086549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:44.086594 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:44.086672 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:44.087431 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:44.087960 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:44.089126 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:44.089333 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:44.089420 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:44.092198 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:44.092202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:44.092206 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:44.092231 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:44.094269 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:44.094746 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:44.094752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:44.094754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:44.094843 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:44.095151 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:44.095154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:44.095156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:44.095361 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:44.095502 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:44.095785 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 09:40:44.095917 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 09:40:44.098881 12430 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:40:44.099344 12430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:44.104635 12593 tablet_service.cc:1467] Processing CreateTablet for tablet a3c99e651d4441cc9747510b7a87f38b (DEFAULT_TABLE table=test-table [id=a2ae9e60ad584002ae0a5b2d3e7545be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:44.104801 12749 tablet_service.cc:1467] Processing CreateTablet for tablet a3c99e651d4441cc9747510b7a87f38b (DEFAULT_TABLE table=test-table [id=a2ae9e60ad584002ae0a5b2d3e7545be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:44.104957 12593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c99e651d4441cc9747510b7a87f38b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:44.105062 12749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c99e651d4441cc9747510b7a87f38b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:44.107395 13025 tablet_bootstrap.cc:492] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: Bootstrap starting.
I20260812 09:40:44.107417 13026 tablet_bootstrap.cc:492] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: Bootstrap starting.
I20260812 09:40:44.107556 12916 tablet_service.cc:1467] Processing CreateTablet for tablet a3c99e651d4441cc9747510b7a87f38b (DEFAULT_TABLE table=test-table [id=a2ae9e60ad584002ae0a5b2d3e7545be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:44.107838 12916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c99e651d4441cc9747510b7a87f38b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:44.108327 13026 tablet_bootstrap.cc:654] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:44.108446 13025 tablet_bootstrap.cc:654] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:44.109988 13025 tablet_bootstrap.cc:492] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: No bootstrap required, opened a new log
I20260812 09:40:44.110088 13025 ts_tablet_manager.cc:1397] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:44.110239 13023 tablet_bootstrap.cc:492] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: Bootstrap starting.
I20260812 09:40:44.110563 13025 raft_consensus.cc:348] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.110687 13025 raft_consensus.cc:374] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:44.110723 13025 raft_consensus.cc:729] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d9aa44630d479aa34ef0db5965ccb8, State: Initialized, Role: FOLLOWER
I20260812 09:40:44.110844 13025 consensus_queue.cc:260] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.111078 13023 tablet_bootstrap.cc:654] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:44.111999 13025 ts_tablet_manager.cc:1428] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 09:40:44.116982 13026 tablet_bootstrap.cc:492] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: No bootstrap required, opened a new log
I20260812 09:40:44.117077 13026 ts_tablet_manager.cc:1397] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20260812 09:40:44.117113 13023 tablet_bootstrap.cc:492] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: No bootstrap required, opened a new log
I20260812 09:40:44.117187 13023 ts_tablet_manager.cc:1397] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260812 09:40:44.117533 13023 raft_consensus.cc:348] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.117625 13023 raft_consensus.cc:374] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:44.117652 13023 raft_consensus.cc:729] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58c84661a0548aab8e8ffe18677c971, State: Initialized, Role: FOLLOWER
I20260812 09:40:44.117692 13026 raft_consensus.cc:348] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.117775 13023 consensus_queue.cc:260] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.117818 13026 raft_consensus.cc:374] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:44.117854 13026 raft_consensus.cc:729] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979027e14752410a820ad5c6f4bc14a5, State: Initialized, Role: FOLLOWER
I20260812 09:40:44.117997 13023 ts_tablet_manager.cc:1428] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:44.117980 13026 consensus_queue.cc:260] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.118247 13026 ts_tablet_manager.cc:1428] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:44.365983 13050 raft_consensus.cc:482] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:44.366173 13050 raft_consensus.cc:504] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.366605 13050 leader_election.cc:288] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963), 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993)
I20260812 09:40:44.367183 12617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c99e651d4441cc9747510b7a87f38b" candidate_uuid: "d58c84661a0548aab8e8ffe18677c971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d9aa44630d479aa34ef0db5965ccb8" is_pre_election: true
I20260812 09:40:44.367390 12617 raft_consensus.cc:2454] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58c84661a0548aab8e8ffe18677c971 in term 0.
I20260812 09:40:44.367354 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c99e651d4441cc9747510b7a87f38b" candidate_uuid: "d58c84661a0548aab8e8ffe18677c971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5" is_pre_election: true
I20260812 09:40:44.367539 12770 raft_consensus.cc:2454] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58c84661a0548aab8e8ffe18677c971 in term 0.
I20260812 09:40:44.367826 12860 leader_election.cc:302] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, d58c84661a0548aab8e8ffe18677c971; no voters:
I20260812 09:40:44.368119 13050 raft_consensus.cc:2781] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:44.368216 13050 raft_consensus.cc:482] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:44.368253 13050 raft_consensus.cc:3037] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:44.369547 13050 raft_consensus.cc:504] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.369890 13050 leader_election.cc:288] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [CANDIDATE]: Term 1 election: Requested vote from peers 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963), 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993)
I20260812 09:40:44.370334 12617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c99e651d4441cc9747510b7a87f38b" candidate_uuid: "d58c84661a0548aab8e8ffe18677c971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d9aa44630d479aa34ef0db5965ccb8"
I20260812 09:40:44.370450 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c99e651d4441cc9747510b7a87f38b" candidate_uuid: "d58c84661a0548aab8e8ffe18677c971" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5"
I20260812 09:40:44.370500 12617 raft_consensus.cc:3037] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:44.370633 12770 raft_consensus.cc:3037] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:44.371596 12617 raft_consensus.cc:2454] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58c84661a0548aab8e8ffe18677c971 in term 1.
I20260812 09:40:44.371780 12770 raft_consensus.cc:2454] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58c84661a0548aab8e8ffe18677c971 in term 1.
I20260812 09:40:44.371941 12860 leader_election.cc:302] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, d58c84661a0548aab8e8ffe18677c971; no voters:
I20260812 09:40:44.372170 13050 raft_consensus.cc:2781] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:44.372422 13050 raft_consensus.cc:686] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 1 LEADER]: Becoming Leader. State: Replica: d58c84661a0548aab8e8ffe18677c971, State: Running, Role: LEADER
I20260812 09:40:44.372601 13050 consensus_queue.cc:237] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.374344 12430 catalog_manager.cc:5495] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 reported cstate change: term changed from 0 to 1, leader changed from <none> to d58c84661a0548aab8e8ffe18677c971 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "d58c84661a0548aab8e8ffe18677c971" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } health_report { overall_health: HEALTHY } } }
I20260812 09:40:44.409400 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:44.406038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.3.147.3:45383)
0812 09:40:44.406373 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:44.406377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:44.406394 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:40:44.406448 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:44.406451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:44.406502 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:44.406590 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:44.406830 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:44.407427 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:44.408395 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:44.408659 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:44.408755 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:44.409002 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:44.409082 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:44.409160 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:44.409232 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 09:40:44.409279 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":158,"threads_started":1}
I20260812 09:40:44.411535 12617 raft_consensus.cc:1261] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Refusing update from remote peer d58c84661a0548aab8e8ffe18677c971: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:44.411541 12770 raft_consensus.cc:1261] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Refusing update from remote peer d58c84661a0548aab8e8ffe18677c971: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:44.412014 13050 consensus_queue.cc:1035] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:44.412240 13101 consensus_queue.cc:1035] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:44.416020 13043 mvcc.cc:205] Tried to move back new op lower bound from 7317617231504916480 to 7317617231351504896. Current Snapshot: MvccSnapshot[applied={T|T < 7317617231504916480}]
I20260812 09:40:44.420634 12430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:44.421002 12430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:44.423298 12430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: Sending DeleteTablet for 3 replicas of tablet a3c99e651d4441cc9747510b7a87f38b
I20260812 09:40:44.423883 12593 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c99e651d4441cc9747510b7a87f38b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:48540
I20260812 09:40:44.423949 12749 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c99e651d4441cc9747510b7a87f38b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:53870
I20260812 09:40:44.423945 12916 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c99e651d4441cc9747510b7a87f38b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:59800
I20260812 09:40:44.424129 13053 tablet_replica.cc:330] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: stopping tablet replica
I20260812 09:40:44.424124 13055 tablet_replica.cc:330] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: stopping tablet replica
I20260812 09:40:44.424125 13054 tablet_replica.cc:330] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: stopping tablet replica
I20260812 09:40:44.424265 13055 raft_consensus.cc:2229] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:44.424266 13054 raft_consensus.cc:2229] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:44.424273 13053 raft_consensus.cc:2229] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:44.424386 13055 raft_consensus.cc:2258] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.424386 13054 raft_consensus.cc:2258] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.424465 13053 raft_consensus.cc:2258] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.425216 13053 ts_tablet_manager.cc:1905] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:44.425303 13055 ts_tablet_manager.cc:1905] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:44.425390 12430 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 09:40:44.425661 13054 ts_tablet_manager.cc:1905] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 09:40:44.425786 12430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:40:44.427924 13055 ts_tablet_manager.cc:1918] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:44.428010 13055 log.cc:1196] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/a3c99e651d4441cc9747510b7a87f38b
I20260812 09:40:44.427996 13053 ts_tablet_manager.cc:1918] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:44.428086 13053 log.cc:1196] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/a3c99e651d4441cc9747510b7a87f38b
I20260812 09:40:44.428269 13054 ts_tablet_manager.cc:1918] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:40:44.428336 13055 ts_tablet_manager.cc:1939] T a3c99e651d4441cc9747510b7a87f38b P 979027e14752410a820ad5c6f4bc14a5: Deleting consensus metadata
I20260812 09:40:44.428350 13054 log.cc:1196] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/a3c99e651d4441cc9747510b7a87f38b
I20260812 09:40:44.428422 13053 ts_tablet_manager.cc:1939] T a3c99e651d4441cc9747510b7a87f38b P d58c84661a0548aab8e8ffe18677c971: Deleting consensus metadata
I20260812 09:40:44.428640 13054 ts_tablet_manager.cc:1939] T a3c99e651d4441cc9747510b7a87f38b P 35d9aa44630d479aa34ef0db5965ccb8: Deleting consensus metadata
I20260812 09:40:44.429148 12415 catalog_manager.cc:4841] TS d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383): tablet a3c99e651d4441cc9747510b7a87f38b (table test-table [id=a2ae9e60ad584002ae0a5b2d3e7545be]) successfully deleted
I20260812 09:40:44.429281 12417 catalog_manager.cc:4841] TS 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963): tablet a3c99e651d4441cc9747510b7a87f38b (table test-table [id=a2ae9e60ad584002ae0a5b2d3e7545be]) successfully deleted
I20260812 09:40:44.429432 12417 catalog_manager.cc:4841] TS 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993): tablet a3c99e651d4441cc9747510b7a87f38b (table test-table [id=a2ae9e60ad584002ae0a5b2d3e7545be]) successfully deleted
I20260812 09:40:44.430891 12593 tablet_service.cc:1467] Processing CreateTablet for tablet 515cb0b624a1489c97a6079c6dbd67af (DEFAULT_TABLE table=test-table [id=3c8443c2801f4675a18754cde0c5ac38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:44.430955 12749 tablet_service.cc:1467] Processing CreateTablet for tablet 515cb0b624a1489c97a6079c6dbd67af (DEFAULT_TABLE table=test-table [id=3c8443c2801f4675a18754cde0c5ac38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:44.431113 12916 tablet_service.cc:1467] Processing CreateTablet for tablet 515cb0b624a1489c97a6079c6dbd67af (DEFAULT_TABLE table=test-table [id=3c8443c2801f4675a18754cde0c5ac38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:40:44.431181 12593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 515cb0b624a1489c97a6079c6dbd67af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:44.431205 12749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 515cb0b624a1489c97a6079c6dbd67af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:44.431380 12916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 515cb0b624a1489c97a6079c6dbd67af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:44.433105 13023 tablet_bootstrap.cc:492] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: Bootstrap starting.
I20260812 09:40:44.433440 13026 tablet_bootstrap.cc:492] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: Bootstrap starting.
I20260812 09:40:44.433930 13023 tablet_bootstrap.cc:654] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:44.434401 13026 tablet_bootstrap.cc:654] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:44.435012 13025 tablet_bootstrap.cc:492] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8: Bootstrap starting.
I20260812 09:40:44.435187 13023 tablet_bootstrap.cc:492] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: No bootstrap required, opened a new log
I20260812 09:40:44.435271 13023 ts_tablet_manager.cc:1397] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 09:40:44.435655 13023 raft_consensus.cc:348] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.435794 13023 raft_consensus.cc:374] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:44.435853 13025 tablet_bootstrap.cc:654] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8: Neither blocks nor log segments found. Creating new log.
I20260812 09:40:44.435846 13023 raft_consensus.cc:729] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58c84661a0548aab8e8ffe18677c971, State: Initialized, Role: FOLLOWER
I20260812 09:40:44.435976 13023 consensus_queue.cc:260] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.436306 13023 ts_tablet_manager.cc:1428] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260812 09:40:44.436707 13026 tablet_bootstrap.cc:492] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: No bootstrap required, opened a new log
I20260812 09:40:44.436822 13026 ts_tablet_manager.cc:1397] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 09:40:44.437181 13026 raft_consensus.cc:348] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.437269 13026 raft_consensus.cc:374] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:44.437291 13026 raft_consensus.cc:729] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979027e14752410a820ad5c6f4bc14a5, State: Initialized, Role: FOLLOWER
I20260812 09:40:44.437417 13026 consensus_queue.cc:260] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.437690 13026 ts_tablet_manager.cc:1428] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 09:40:44.438673 13025 tablet_bootstrap.cc:492] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8: No bootstrap required, opened a new log
I20260812 09:40:44.438763 13025 ts_tablet_manager.cc:1397] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 09:40:44.439241 13025 raft_consensus.cc:348] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.439353 13025 raft_consensus.cc:374] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:44.439388 13025 raft_consensus.cc:729] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d9aa44630d479aa34ef0db5965ccb8, State: Initialized, Role: FOLLOWER
I20260812 09:40:44.439498 13025 consensus_queue.cc:260] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.439805 13025 ts_tablet_manager.cc:1428] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 09:40:44.442225 13033 raft_consensus.cc:482] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:40:44.442322 13033 raft_consensus.cc:504] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.442699 13033 leader_election.cc:288] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993), d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383)
I20260812 09:40:44.442997 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "515cb0b624a1489c97a6079c6dbd67af" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5" is_pre_election: true
I20260812 09:40:44.443027 12936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "515cb0b624a1489c97a6079c6dbd67af" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58c84661a0548aab8e8ffe18677c971" is_pre_election: true
I20260812 09:40:44.443145 12936 raft_consensus.cc:2454] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 0.
I20260812 09:40:44.443158 12770 raft_consensus.cc:2454] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 0.
I20260812 09:40:44.443388 12536 leader_election.cc:302] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, d58c84661a0548aab8e8ffe18677c971; no voters:
I20260812 09:40:44.443564 13033 raft_consensus.cc:2781] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:40:44.443624 13033 raft_consensus.cc:482] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:40:44.443668 13033 raft_consensus.cc:3037] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:44.444576 13033 raft_consensus.cc:504] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.444922 13033 leader_election.cc:288] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 election: Requested vote from peers 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993), d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383)
I20260812 09:40:44.445272 12936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "515cb0b624a1489c97a6079c6dbd67af" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58c84661a0548aab8e8ffe18677c971"
I20260812 09:40:44.445272 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "515cb0b624a1489c97a6079c6dbd67af" candidate_uuid: "35d9aa44630d479aa34ef0db5965ccb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979027e14752410a820ad5c6f4bc14a5"
I20260812 09:40:44.445400 12770 raft_consensus.cc:3037] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:44.445400 12936 raft_consensus.cc:3037] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:44.446522 12770 raft_consensus.cc:2454] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 1.
I20260812 09:40:44.446532 12936 raft_consensus.cc:2454] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d9aa44630d479aa34ef0db5965ccb8 in term 1.
I20260812 09:40:44.446815 12536 leader_election.cc:302] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d9aa44630d479aa34ef0db5965ccb8, d58c84661a0548aab8e8ffe18677c971; no voters:
I20260812 09:40:44.446996 13033 raft_consensus.cc:2781] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:44.447110 13033 raft_consensus.cc:686] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 LEADER]: Becoming Leader. State: Replica: 35d9aa44630d479aa34ef0db5965ccb8, State: Running, Role: LEADER
I20260812 09:40:44.447249 13033 consensus_queue.cc:237] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } }
I20260812 09:40:44.448689 12429 catalog_manager.cc:5495] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1). New cstate: current_term: 1 leader_uuid: "35d9aa44630d479aa34ef0db5965ccb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35d9aa44630d479aa34ef0db5965ccb8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 36963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 } health_report { overall_health: UNKNOWN } } }
I20260812 09:40:44.462859 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:44.459669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.3.147.1:36963)
0812 09:40:44.459787 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:44.459792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:44.459838 (+ 46us) server_negotiation.cc:407] Connection header received
0812 09:40:44.460002 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:44.460005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:44.460049 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:44.460138 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:40:44.460411 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:44.460930 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:44.461775 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:44.462011 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:44.462118 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:44.462383 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:40:44.462460 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:40:44.462542 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:44.462633 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 09:40:44.462693 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:40:44.468741 12769 raft_consensus.cc:1261] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Refusing update from remote peer 35d9aa44630d479aa34ef0db5965ccb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:44.468742 12936 raft_consensus.cc:1261] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Refusing update from remote peer 35d9aa44630d479aa34ef0db5965ccb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:40:44.469177 13082 consensus_queue.cc:1035] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979027e14752410a820ad5c6f4bc14a5" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 33993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:44.469319 13033 consensus_queue.cc:1035] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58c84661a0548aab8e8ffe18677c971" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:40:44.502197 12429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:44.502365 12429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:40:44.504527 12429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 681a506d6a124c0b89626253b922599a: Sending DeleteTablet for 3 replicas of tablet 515cb0b624a1489c97a6079c6dbd67af
I20260812 09:40:44.505191 12916 tablet_service.cc:1514] Processing DeleteTablet for tablet 515cb0b624a1489c97a6079c6dbd67af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:59800
I20260812 09:40:44.505204 12749 tablet_service.cc:1514] Processing DeleteTablet for tablet 515cb0b624a1489c97a6079c6dbd67af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:53870
I20260812 09:40:44.505225 12593 tablet_service.cc:1514] Processing DeleteTablet for tablet 515cb0b624a1489c97a6079c6dbd67af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:40:44 UTC) from {username='oryx', principal='oryx/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:48540
I20260812 09:40:44.505388 13053 tablet_replica.cc:330] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: stopping tablet replica
I20260812 09:40:44.505462 13054 tablet_replica.cc:330] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8: stopping tablet replica
I20260812 09:40:44.505532 13053 raft_consensus.cc:2229] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:44.505537 13055 tablet_replica.cc:330] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: stopping tablet replica
I20260812 09:40:44.505587 13054 raft_consensus.cc:2229] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:40:44.505649 13053 raft_consensus.cc:2258] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.505743 13055 raft_consensus.cc:2229] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:40:44.505779 13054 raft_consensus.cc:2258] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.505887 13055 raft_consensus.cc:2258] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:40:44.506381 13053 ts_tablet_manager.cc:1905] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:44.506392 13054 ts_tablet_manager.cc:1905] T 515cb0b624a1489c97a6079c6dbd67af P 35d9aa44630d479aa34ef0db5965ccb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:44.506866 13055 ts_tablet_manager.cc:1905] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:40:44.506961 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 12519
I20260812 09:40:44.509225 13053 ts_tablet_manager.cc:1918] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:44.509295 13053 log.cc:1196] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/515cb0b624a1489c97a6079c6dbd67af
I20260812 09:40:44.509521 13053 ts_tablet_manager.cc:1939] T 515cb0b624a1489c97a6079c6dbd67af P d58c84661a0548aab8e8ffe18677c971: Deleting consensus metadata
I20260812 09:40:44.510258 12415 catalog_manager.cc:4841] TS d58c84661a0548aab8e8ffe18677c971 (127.3.147.3:45383): tablet 515cb0b624a1489c97a6079c6dbd67af (table test-table [id=3c8443c2801f4675a18754cde0c5ac38]) successfully deleted
I20260812 09:40:44.513513 13055 ts_tablet_manager.cc:1918] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 09:40:44.513648 13055 log.cc:1196] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/515cb0b624a1489c97a6079c6dbd67af
I20260812 09:40:44.513954 13055 ts_tablet_manager.cc:1939] T 515cb0b624a1489c97a6079c6dbd67af P 979027e14752410a820ad5c6f4bc14a5: Deleting consensus metadata
I20260812 09:40:44.514740 12417 catalog_manager.cc:4841] TS 979027e14752410a820ad5c6f4bc14a5 (127.3.147.2:33993): tablet 515cb0b624a1489c97a6079c6dbd67af (table test-table [id=3c8443c2801f4675a18754cde0c5ac38]) successfully deleted
W20260812 09:40:44.519627 12417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:36963 (error 108)
I20260812 09:40:44.519824 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 12681
I20260812 09:40:44.523545 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:44.523070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:36963 (local address 127.0.0.1:48582)
0812 09:40:44.523320 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:44.523406 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
W20260812 09:40:44.523669 12417 catalog_manager.cc:4584] TS 35d9aa44630d479aa34ef0db5965ccb8 (127.3.147.1:36963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 515cb0b624a1489c97a6079c6dbd67af: Network error: Client connection negotiation failed: client connection to 127.3.147.1:36963: connect: Connection refused (error 111)
I20260812 09:40:44.529517 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 12840
I20260812 09:40:44.538926 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 12396
2026-08-12T09:40:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3620 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:44 dist-test-slave-cbbz krb5kdc[13119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:44 dist-test-slave-cbbz krb5kdc[13119](info): set up 2 sockets
Aug 12 09:40:44 dist-test-slave-cbbz krb5kdc[13119](info): commencing operation
krb5kdc: starting...
W20260812 09:40:45.666302 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:45 dist-test-slave-cbbz krb5kdc[13119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:45.794620 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:34977
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:39369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:34977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:45.906893 13136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:45.907160 13136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:45.907199 13136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:45.911078 13136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:45.911129 13136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:45.911149 13136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:45.911165 13136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:45.911180 13136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:45.916414 13136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:34977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:34977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:45.917379 13136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:45.918404 13136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:45.926926 13145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:45.926919 13141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:45.927058 13136 server_base.cc:1034] running on GCE node
W20260812 09:40:45.926928 13143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:45.927835 13136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:45.929127 13136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:45.930372 13136 hybrid_clock.cc:648] HybridClock initialized: now 1786527645930340 us; error 41 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T09:40:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1399 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:40:46 dist-test-slave-cbbz krb5kdc[13151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:40:46 dist-test-slave-cbbz krb5kdc[13151](info): set up 2 sockets
Aug 12 09:40:46 dist-test-slave-cbbz krb5kdc[13151](info): commencing operation
krb5kdc: starting...
W20260812 09:40:47.067396 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:40:47 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:40:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:40:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:40:47.195983 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35585
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:35585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:47.306695 13166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:47.306993 13166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:47.307071 13166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:47.311061 13166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:47.311152 13166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:47.311190 13166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:47.311226 13166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:47.311257 13166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:47.316427 13166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:35585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:47.317456 13166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:47.318475 13166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:47.326923 13171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:47.326977 13166 server_base.cc:1034] running on GCE node
W20260812 09:40:47.326944 13174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:47.327054 13172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:47.327777 13166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:47.329036 13166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:47.330245 13166 hybrid_clock.cc:648] HybridClock initialized: now 1786527647330218 us; error 45 us; skew 500 ppm
Aug 12 09:40:47 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527647, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:47.333318 13166 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:47.334280 13166 webserver.cc:460] Webserver started at http://127.3.147.62:40415/ using document root <none> and password file <none>
I20260812 09:40:47.335080 13166 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:47.335199 13166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:47.335536 13166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:47.337744 13166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "e3ae58eee0be41448bd44c5ea56076ee"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "36d0fab9700bdf16bab0a520992c5e5d"
server_key_iv: "e35fb2789e9fcb08865b1eec3060400b"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.338312 13166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "e3ae58eee0be41448bd44c5ea56076ee"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "36d0fab9700bdf16bab0a520992c5e5d"
server_key_iv: "e35fb2789e9fcb08865b1eec3060400b"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.342476 13166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:40:47.345044 13184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:47.346323 13166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:47.346472 13166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "e3ae58eee0be41448bd44c5ea56076ee"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "36d0fab9700bdf16bab0a520992c5e5d"
server_key_iv: "e35fb2789e9fcb08865b1eec3060400b"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.346583 13166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:47.357584 13166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:47.360668 13166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:47.360822 13166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:47.367944 13166 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:35585
I20260812 09:40:47.368036 13241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:35585 every 8 connection(s)
I20260812 09:40:47.369197 13166 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:47.371925 13243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:40:47.372080 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 13166
I20260812 09:40:47.372239 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:40:47.372571 3660 external_mini_cluster.cc:1428] Setting key 1cfad0935a21f53c909a8f0ab3067477
I20260812 09:40:47.379350 13243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Bootstrap starting.
I20260812 09:40:47.382048 13243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Neither blocks nor log segments found. Creating new log.
Aug 12 09:40:47 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:47.382942 13243 log.cc:824] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:47.385321 13243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: No bootstrap required, opened a new log
I20260812 09:40:47.388525 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:47.373919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.3.147.62:35585)
0812 09:40:47.374338 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:47.374360 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:47.374404 (+ 44us) server_negotiation.cc:407] Connection header received
0812 09:40:47.375576 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:47.375601 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:47.376013 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:47.376409 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:47.377109 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.378596 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.379725 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.380434 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.380618 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:47.383274 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:47.383283 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:47.383303 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:47.383338 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:47.385313 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:47.385799 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:47.385805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:47.385810 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:47.385950 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:47.386268 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:47.386271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:47.386273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:47.386759 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:47.386941 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:47.387169 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 09:40:47.387500 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":125,"threads_started":1}
I20260812 09:40:47.389842 13243 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } }
I20260812 09:40:47.390084 13243 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:40:47.390142 13243 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ae58eee0be41448bd44c5ea56076ee, State: Initialized, Role: FOLLOWER
I20260812 09:40:47.390606 13243 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } }
I20260812 09:40:47.390771 13243 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:47.390865 13243 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:47.390995 13243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:40:47.391853 13243 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } }
I20260812 09:40:47.392241 13243 leader_election.cc:302] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3ae58eee0be41448bd44c5ea56076ee; no voters:
I20260812 09:40:47.392537 13243 leader_election.cc:288] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:40:47.392777 13249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:40:47.393075 13249 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 1 LEADER]: Becoming Leader. State: Replica: e3ae58eee0be41448bd44c5ea56076ee, State: Running, Role: LEADER
I20260812 09:40:47.393430 13249 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } }
I20260812 09:40:47.393738 13243 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:47.395015 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ae58eee0be41448bd44c5ea56076ee. Latest consensus state: current_term: 1 leader_uuid: "e3ae58eee0be41448bd44c5ea56076ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } } }
I20260812 09:40:47.395162 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:47.395143 13250 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3ae58eee0be41448bd44c5ea56076ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } } }
I20260812 09:40:47.395591 13258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:47.395602 13250 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:47.398936 13258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:47.404362 13258 catalog_manager.cc:1347] Generated new cluster ID: 95df0f7a93fb44cf9a9a26aa97386dbd
I20260812 09:40:47.404778 13258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:47.416926 13258 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:40:47.417814 13258 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:47.434139 13258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Generated new TSK 0
I20260812 09:40:47.434890 13258 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:40:47.467696 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35585
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:47.580986 13273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:47.581235 13273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:47.581303 13273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:47.585379 13273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:47.585433 13273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:47.585518 13273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:47.590616 13273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:47.591648 13273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:47.592717 13273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:47.603111 13280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:47.603189 13286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:47.603207 13273 server_base.cc:1034] running on GCE node
W20260812 09:40:47.603168 13281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:47.603890 13273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:47.604534 13273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:47.605703 13273 hybrid_clock.cc:648] HybridClock initialized: now 1786527647605685 us; error 31 us; skew 500 ppm
Aug 12 09:40:47 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527647, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:47.608537 13273 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:40:47.609400 13273 webserver.cc:460] Webserver started at http://127.3.147.1:36269/ using document root <none> and password file <none>
I20260812 09:40:47.609942 13273 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:47.610002 13273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:47.610235 13273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:47.612336 13273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "e7eacbc69ffc4a65aee268cfbccade0f"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "e0a54d88cc6bf5056eb142fb5eef2abe"
server_key_iv: "650de2e24e71aa760f6671033da670ca"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.612923 13273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "e7eacbc69ffc4a65aee268cfbccade0f"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "e0a54d88cc6bf5056eb142fb5eef2abe"
server_key_iv: "650de2e24e71aa760f6671033da670ca"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.616688 13273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:40:47.619076 13293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:47.620661 13273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:40:47.620818 13273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "e7eacbc69ffc4a65aee268cfbccade0f"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "e0a54d88cc6bf5056eb142fb5eef2abe"
server_key_iv: "650de2e24e71aa760f6671033da670ca"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.620934 13273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:47.639437 13273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:47.642784 13273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:47.642993 13273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:47.643590 13273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:47.644500 13273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:47.644573 13273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:47.644656 13273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:47.644707 13273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:47.654053 13273 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:32827
I20260812 09:40:47.654080 13420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:32827 every 8 connection(s)
I20260812 09:40:47.654930 13273 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
Aug 12 09:40:47 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527647, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:47.664729 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 13273
I20260812 09:40:47.664870 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:40:47.665201 3660 external_mini_cluster.cc:1428] Setting key ca8f67a2e641df2f449b68d174c50094
I20260812 09:40:47.668954 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:47.656830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:33481 (local address 127.3.147.62:35585)
0812 09:40:47.656967 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:47.656971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:47.657831 (+ 860us) server_negotiation.cc:407] Connection header received
0812 09:40:47.659149 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:47.659152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:47.659212 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:47.659316 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:47.660466 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.661002 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.662149 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.662381 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.662494 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:47.665135 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:47.665138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:47.665141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:47.665169 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:47.666934 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:47.667433 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:47.667438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:47.667440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:47.667519 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:47.667856 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:47.667859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:47.667861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:47.668038 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:47.668172 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:47.668653 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 09:40:47.668773 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:40:47.669726 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:47.657230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:33481)
0812 09:40:47.657677 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:47.657718 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:47.658910 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:47.659485 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:47.659491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:47.659838 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:47.660316 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:47.660330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.661137 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.661141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:47.662017 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:47.662026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.662537 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.662541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:47.662716 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:47.663287 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:47.663303 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:47.664913 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:47.667052 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:47.667058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:47.667072 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:47.667329 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:47.667619 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:47.667622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:47.667624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:47.667759 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:47.668174 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:47.668182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:47.668419 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 09:40:47.668607 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 09:40:47.668922 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260812 09:40:47.671154 13421 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35585
I20260812 09:40:47.671417 13421 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:47.671900 13421 heartbeater.cc:511] Master 127.3.147.62:35585 requested a full tablet report, sending...
I20260812 09:40:47.673588 13205 ts_manager.cc:194] Registered new tserver with Master: e7eacbc69ffc4a65aee268cfbccade0f (127.3.147.1:32827)
I20260812 09:40:47.674728 13205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:33481
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:40:47.696844 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35585
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:47.808379 13426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:47.808633 13426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:47.808676 13426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:47.812606 13426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:47.812664 13426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:47.812719 13426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:47.817786 13426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:47.818769 13426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:47.819743 13426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:47.829293 13432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:47.829380 13434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:47.829290 13436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:47.829479 13426 server_base.cc:1034] running on GCE node
I20260812 09:40:47.829977 13426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:47.830607 13426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:47.831785 13426 hybrid_clock.cc:648] HybridClock initialized: now 1786527647831768 us; error 31 us; skew 500 ppm
Aug 12 09:40:47 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527647, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:47.834780 13426 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:40:47.835677 13426 webserver.cc:460] Webserver started at http://127.3.147.2:46539/ using document root <none> and password file <none>
I20260812 09:40:47.836308 13426 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:47.836428 13426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:47.836668 13426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:47.838660 13426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "aace0fe3b30844c888cfcfefb2f5907d"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "2cd9cea7ebbc8f459429de0624734a90"
server_key_iv: "f6e3c0ba3ecd3dc129098fbb91f92bf0"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.839251 13426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "aace0fe3b30844c888cfcfefb2f5907d"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "2cd9cea7ebbc8f459429de0624734a90"
server_key_iv: "f6e3c0ba3ecd3dc129098fbb91f92bf0"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.843061 13426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 09:40:47.845548 13448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:47.846782 13426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:47.846921 13426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "aace0fe3b30844c888cfcfefb2f5907d"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "2cd9cea7ebbc8f459429de0624734a90"
server_key_iv: "f6e3c0ba3ecd3dc129098fbb91f92bf0"
server_key_version: "encryptionkey@0"
I20260812 09:40:47.847043 13426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:47.866231 13426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:47.869402 13426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:47.869637 13426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:47.870251 13426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:47.871109 13426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:47.871183 13426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:47.871256 13426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:47.871301 13426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:47.880877 13426 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46769
I20260812 09:40:47.880914 13582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46769 every 8 connection(s)
I20260812 09:40:47.881768 13426 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:47.883208 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 13426
I20260812 09:40:47.883354 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:40:47.883708 3660 external_mini_cluster.cc:1428] Setting key 06f3e48dc196a56fbe03f42c0e5960ba
Aug 12 09:40:47 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527647, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:40:47.898379 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:47.883672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:57081 (local address 127.3.147.62:35585)
0812 09:40:47.883832 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:47.883838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:47.884846 (+ 1008us) server_negotiation.cc:407] Connection header received
0812 09:40:47.886783 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:47.886786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:47.886831 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:47.886910 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:47.888472 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.889031 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.890912 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.891227 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.891314 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:47.894495 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:47.894498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:47.894500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:47.894523 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:47.896332 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:47.896949 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:47.896952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:47.896953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:47.897032 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:47.897373 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:47.897376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:47.897377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:47.897550 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:47.897635 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:47.898160 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 09:40:47.898257 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:40:47.899523 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:47.884213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:57081)
0812 09:40:47.884698 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:47.884742 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:47.886485 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:47.887125 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:47.887137 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:47.887695 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:47.888323 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:47.888340 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.889195 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.889201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:47.890768 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:47.890784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:47.891398 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:47.891417 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:47.891672 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:47.892427 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:47.892452 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:47.894316 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:47.896464 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:47.896471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:47.896485 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:47.896829 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:47.897135 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:47.897138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:47.897139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:47.897276 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:47.897640 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:47.897645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:47.897910 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 09:40:47.898094 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 09:40:47.898448 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":121,"threads_started":1}
I20260812 09:40:47.900841 13583 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35585
I20260812 09:40:47.901119 13583 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:47.901767 13583 heartbeater.cc:511] Master 127.3.147.62:35585 requested a full tablet report, sending...
I20260812 09:40:47.902936 13205 ts_manager.cc:194] Registered new tserver with Master: aace0fe3b30844c888cfcfefb2f5907d (127.3.147.2:46769)
I20260812 09:40:47.903604 13205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:57081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:40:47.915774 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35585
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:48.026391 13588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:48.026612 13588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:48.026654 13588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:48.030602 13588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:48.030651 13588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:48.030747 13588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:48.035801 13588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:48.036818 13588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:48.037724 13588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:48.047200 13597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.047178 13593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.047212 13595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:48.047392 13588 server_base.cc:1034] running on GCE node
I20260812 09:40:48.047871 13588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:48.048583 13588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:48.049793 13588 hybrid_clock.cc:648] HybridClock initialized: now 1786527648049750 us; error 59 us; skew 500 ppm
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527648, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:48.052846 13588 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:40:48.053684 13588 webserver.cc:460] Webserver started at http://127.3.147.3:45805/ using document root <none> and password file <none>
I20260812 09:40:48.054201 13588 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:48.054260 13588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:48.054418 13588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:40:48.056480 13588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "8114690a3b8943118fa5c1485b2af3be"
format_stamp: "Formatted at 2026-08-12 09:40:48 on dist-test-slave-cbbz"
server_key: "849198472bde76741d5a15f659ce6fb1"
server_key_iv: "629867d01beed6083ec877b9fcbad422"
server_key_version: "encryptionkey@0"
I20260812 09:40:48.056999 13588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "8114690a3b8943118fa5c1485b2af3be"
format_stamp: "Formatted at 2026-08-12 09:40:48 on dist-test-slave-cbbz"
server_key: "849198472bde76741d5a15f659ce6fb1"
server_key_iv: "629867d01beed6083ec877b9fcbad422"
server_key_version: "encryptionkey@0"
I20260812 09:40:48.060673 13588 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 09:40:48.063140 13605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.064495 13588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:48.064620 13588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "8114690a3b8943118fa5c1485b2af3be"
format_stamp: "Formatted at 2026-08-12 09:40:48 on dist-test-slave-cbbz"
server_key: "849198472bde76741d5a15f659ce6fb1"
server_key_iv: "629867d01beed6083ec877b9fcbad422"
server_key_version: "encryptionkey@0"
I20260812 09:40:48.064723 13588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:48.080274 13588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:48.083530 13588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:48.083735 13588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:48.084357 13588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:48.085162 13588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:48.085211 13588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.085250 13588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:48.085310 13588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.094563 13588 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:34369
I20260812 09:40:48.094645 13740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:34369 every 8 connection(s)
I20260812 09:40:48.095431 13588 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:48.102057 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 13588
I20260812 09:40:48.102205 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:40:48.102484 3660 external_mini_cluster.cc:1428] Setting key aebbb26d01f45c5e37703fdc73e4459b
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527648, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:40:48.109264 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.097330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:33799 (local address 127.3.147.62:35585)
0812 09:40:48.097508 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.097512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.098284 (+ 772us) server_negotiation.cc:407] Connection header received
0812 09:40:48.099624 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.099627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.099690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.099775 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.100984 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.101526 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.102843 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.103143 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.103244 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.105747 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:48.105751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:48.105754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:48.105783 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:48.107134 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:48.107668 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:48.107672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:48.107674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:48.107762 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:48.108157 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:48.108163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:48.108164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:48.108355 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:48.108472 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:48.108960 (+ 488us) server_negotiation.cc:299] Negotiation successful
0812 09:40:48.109084 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 09:40:48.110011 13744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.097627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:33799)
0812 09:40:48.098099 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.098151 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.099397 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.099942 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.099948 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.100338 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.100818 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.100832 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.101677 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.101680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.102715 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.102726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.103286 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.103290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.103457 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.104059 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.104076 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:48.105574 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:40:48.107258 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:48.107264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:48.107277 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:48.107534 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:48.107862 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:40:48.107865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:40:48.107867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:40:48.108009 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:40:48.108472 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:40:48.108480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:40:48.108720 (+ 240us) client_negotiation.cc:769] Sending connection context
0812 09:40:48.108910 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 09:40:48.109229 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":114,"threads_started":1}
I20260812 09:40:48.111110 13741 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35585
I20260812 09:40:48.111354 13741 heartbeater.cc:464] Registering TS with master...
I20260812 09:40:48.111822 13741 heartbeater.cc:511] Master 127.3.147.62:35585 requested a full tablet report, sending...
I20260812 09:40:48.112974 13205 ts_manager.cc:194] Registered new tserver with Master: 8114690a3b8943118fa5c1485b2af3be (127.3.147.3:34369)
I20260812 09:40:48.113690 13205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:33799
I20260812 09:40:48.116343 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab.
Entry for principal oryx/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab.
WARNING: no policy specified for oryx/127.3.147.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab.
Entry for principal oryx/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab.
WARNING: no policy specified for oryx/127.3.147.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab.
Entry for principal oryx/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab.
I20260812 09:40:48.191774 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 13273
I20260812 09:40:48.199931 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 13426
I20260812 09:40:48.210389 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 13588
I20260812 09:40:48.220767 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 13166
I20260812 09:40:48.228683 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35585
--webserver_interface=127.3.147.62
--webserver_port=40415
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:35585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:40:48.338227 13751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:48.338475 13751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:48.338513 13751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:48.342396 13751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:40:48.342445 13751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:48.342464 13751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:48.342480 13751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:40:48.342495 13751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:40:48.347558 13751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:35585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=40415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:48.348526 13751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:48.349478 13751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:48.357909 13758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:48.357942 13751 server_base.cc:1034] running on GCE node
W20260812 09:40:48.357944 13760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.357909 13757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:48.358760 13751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:48.359915 13751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:48.361084 13751 hybrid_clock.cc:648] HybridClock initialized: now 1786527648361054 us; error 54 us; skew 500 ppm
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527648, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:48.363965 13751 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:40:48.364912 13751 webserver.cc:460] Webserver started at http://127.3.147.62:40415/ using document root <none> and password file <none>
I20260812 09:40:48.365458 13751 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:48.365530 13751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:48.369315 13751 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 09:40:48.371438 13768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.372800 13751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:48.372942 13751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "e3ae58eee0be41448bd44c5ea56076ee"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "36d0fab9700bdf16bab0a520992c5e5d"
server_key_iv: "e35fb2789e9fcb08865b1eec3060400b"
server_key_version: "encryptionkey@0"
I20260812 09:40:48.373479 13751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:48.383783 13751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:48.386693 13751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:48.386894 13751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:48.394177 13751 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:35585
I20260812 09:40:48.394258 13828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:35585 every 8 connection(s)
I20260812 09:40:48.395254 13751 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:40:48.399830 13829 sys_catalog.cc:263] Verifying existing consensus state
I20260812 09:40:48.401646 13829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Bootstrap starting.
I20260812 09:40:48.404542 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 13751
I20260812 09:40:48.404979 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:32827
--local_ip_for_outbound_sockets=127.3.147.1
--tserver_master_addrs=127.3.147.62:35585
--webserver_port=36269
--webserver_interface=127.3.147.1
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.147.1
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:40:48.421195 13829 log.cc:824] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Log is configured to *not* fsync() on all Append() calls
I20260812 09:40:48.428551 13829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:40:48.428990 13829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Bootstrap complete.
I20260812 09:40:48.433822 13829 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } }
I20260812 09:40:48.434254 13829 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ae58eee0be41448bd44c5ea56076ee, State: Initialized, Role: FOLLOWER
I20260812 09:40:48.434796 13829 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } }
I20260812 09:40:48.434921 13829 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:40:48.435021 13829 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:40:48.435129 13829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 1 FOLLOWER]: Advancing to term 2
I20260812 09:40:48.437340 13829 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } }
I20260812 09:40:48.437709 13829 leader_election.cc:302] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3ae58eee0be41448bd44c5ea56076ee; no voters:
I20260812 09:40:48.438082 13829 leader_election.cc:288] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 09:40:48.438198 13833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 2 FOLLOWER]: Leader election won for term 2
I20260812 09:40:48.438443 13833 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [term 2 LEADER]: Becoming Leader. State: Replica: e3ae58eee0be41448bd44c5ea56076ee, State: Running, Role: LEADER
I20260812 09:40:48.438832 13833 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } }
I20260812 09:40:48.439451 13829 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:40:48.440469 13835 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ae58eee0be41448bd44c5ea56076ee. Latest consensus state: current_term: 2 leader_uuid: "e3ae58eee0be41448bd44c5ea56076ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } } }
I20260812 09:40:48.440478 13834 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e3ae58eee0be41448bd44c5ea56076ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3ae58eee0be41448bd44c5ea56076ee" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35585 } } }
I20260812 09:40:48.440656 13834 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:48.440656 13835 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee [sys.catalog]: This master's current role is: LEADER
I20260812 09:40:48.441133 13844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:40:48.444828 13844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:40:48.445593 13844 catalog_manager.cc:1259] Loaded cluster ID: 95df0f7a93fb44cf9a9a26aa97386dbd
I20260812 09:40:48.445665 13844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:40:48.448297 13844 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:40:48.449155 13844 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e3ae58eee0be41448bd44c5ea56076ee: Loaded TSK: 0
I20260812 09:40:48.450466 13844 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 09:40:48.521667 13831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:48.521951 13831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:48.522025 13831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:48.526007 13831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:48.526086 13831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:48.526175 13831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:40:48.531325 13831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.1.keytab
--principal=oryx/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:32827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=36269
--tserver_master_addrs=127.3.147.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:48.532352 13831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:48.533324 13831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:48.543870 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.543926 13864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.543924 13861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:48.544510 13831 server_base.cc:1034] running on GCE node
I20260812 09:40:48.544901 13831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:48.545509 13831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:48.546716 13831 hybrid_clock.cc:648] HybridClock initialized: now 1786527648546684 us; error 50 us; skew 500 ppm
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527648, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:48.549574 13831 init.cc:377] Logged in from keytab as oryx/127.3.147.1@KRBTEST.COM (short username oryx)
I20260812 09:40:48.550532 13831 webserver.cc:460] Webserver started at http://127.3.147.1:36269/ using document root <none> and password file <none>
I20260812 09:40:48.551096 13831 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:48.551218 13831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:48.555382 13831 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:48.557565 13871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.558851 13831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:40:48.558943 13831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "e7eacbc69ffc4a65aee268cfbccade0f"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "e0a54d88cc6bf5056eb142fb5eef2abe"
server_key_iv: "650de2e24e71aa760f6671033da670ca"
server_key_version: "encryptionkey@0"
I20260812 09:40:48.559494 13831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:48.583138 13831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:48.586527 13831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:48.586747 13831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:48.587378 13831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:48.588353 13831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:48.588402 13831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.588508 13831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:48.588593 13831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.598016 13831 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:32827
I20260812 09:40:48.598055 14000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:32827 every 8 connection(s)
I20260812 09:40:48.599043 13831 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:40:48.602176 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 13831
I20260812 09:40:48.602787 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:46769
--local_ip_for_outbound_sockets=127.3.147.2
--tserver_master_addrs=127.3.147.62:35585
--webserver_port=46539
--webserver_interface=127.3.147.2
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.147.2
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:48.614322 14005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.601181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:50353)
0812 09:40:48.601658 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.601700 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.603457 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.604916 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.604926 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.605440 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.606128 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.606152 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.608146 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.608149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.609051 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.609062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.610182 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.610185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.610319 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.610948 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.610964 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:48.613617 (+ 2653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":133,"threads_started":1}
W20260812 09:40:48.614845 14002 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:35585 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.147.62:35585: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
W20260812 09:40:48.615435 14004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:48.601109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:50353 (local address 127.3.147.62:35585)
0812 09:40:48.601590 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.601625 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.601828 (+ 203us) server_negotiation.cc:407] Connection header received
0812 09:40:48.603892 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.603908 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.604264 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.604627 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.606326 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.607976 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.609184 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.610004 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.610218 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.613901 (+ 3683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:50353: BlockingRecv error: recv got EOF from 127.3.147.1:50353 (error 108)
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":104,"threads_started":1}
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:48.620687 14005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.615336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:41661)
0812 09:40:48.615500 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.615706 (+ 206us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.615794 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.616062 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.616066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.616262 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.616347 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.616353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.617081 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.617086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.617617 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.617625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.618122 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.618126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.618227 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.618665 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.618679 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:48.620546 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260812 09:40:48.620992 14007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:48.615415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:41661 (local address 127.3.147.62:35585)
0812 09:40:48.615706 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.615715 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.615757 (+ 42us) server_negotiation.cc:407] Connection header received
0812 09:40:48.615904 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.615907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.615953 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.616047 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.616471 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.616970 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.617741 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.617974 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.618103 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.620669 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:41661: BlockingRecv error: recv got EOF from 127.3.147.1:41661 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":106,"threads_started":1}
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:48.626441 14005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.621160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:37445)
0812 09:40:48.621285 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.621300 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.621383 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.621610 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.621616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.621793 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.621918 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.621925 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.622657 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.622660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.623505 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.623515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.623990 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.623997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.624088 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.624527 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.624541 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:48.626314 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 09:40:48.626550 14007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:48.621309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:37445 (local address 127.3.147.62:35585)
0812 09:40:48.621418 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.621421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.621433 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:40:48.621461 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.621463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.621496 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.621561 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.622060 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.622532 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.623630 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.623870 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.623953 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.626438 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:37445: BlockingRecv error: recv got EOF from 127.3.147.1:37445 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 09:40:48.626732 14002 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 09:40:48.724850 14006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:48.725116 14006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:48.725162 14006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:48.729076 14006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:48.729133 14006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:48.729190 14006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:40:48.734385 14006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.2.keytab
--principal=oryx/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:46769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=46539
--tserver_master_addrs=127.3.147.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:48.735381 14006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:48.736400 14006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:48.746572 14013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.746572 14012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.746651 14015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:48.747176 14006 server_base.cc:1034] running on GCE node
I20260812 09:40:48.747593 14006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:48.748432 14006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:48.749588 14006 hybrid_clock.cc:648] HybridClock initialized: now 1786527648749570 us; error 33 us; skew 500 ppm
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527648, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:48.752650 14006 init.cc:377] Logged in from keytab as oryx/127.3.147.2@KRBTEST.COM (short username oryx)
I20260812 09:40:48.753961 14006 webserver.cc:460] Webserver started at http://127.3.147.2:46539/ using document root <none> and password file <none>
I20260812 09:40:48.754804 14006 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:48.754905 14006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:48.759670 14006 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:48.761953 14023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.763375 14006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:40:48.763515 14006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "aace0fe3b30844c888cfcfefb2f5907d"
format_stamp: "Formatted at 2026-08-12 09:40:47 on dist-test-slave-cbbz"
server_key: "2cd9cea7ebbc8f459429de0624734a90"
server_key_iv: "f6e3c0ba3ecd3dc129098fbb91f92bf0"
server_key_version: "encryptionkey@0"
I20260812 09:40:48.764091 14006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:48.781086 14006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:48.784255 14006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:48.784451 14006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:48.785059 14006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:48.785892 14006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:48.785966 14006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.786034 14006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:48.786082 14006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.795194 14006 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46769
I20260812 09:40:48.795215 14149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46769 every 8 connection(s)
I20260812 09:40:48.796253 14006 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:40:48.801075 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 14006
I20260812 09:40:48.801530 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:34369
--local_ip_for_outbound_sockets=127.3.147.3
--tserver_master_addrs=127.3.147.62:35585
--webserver_port=45805
--webserver_interface=127.3.147.3
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.147.3
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 09:40:48.807591 14007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:48.797965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:54049 (local address 127.3.147.62:35585)
0812 09:40:48.798106 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.798109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.798913 (+ 804us) server_negotiation.cc:407] Connection header received
0812 09:40:48.800305 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.800308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.800373 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.800482 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.801672 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.802242 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.803399 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.803642 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.803777 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.807419 (+ 3642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:54049: BlockingRecv error: recv got EOF from 127.3.147.2:54049 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 09:40:48.808010 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.798351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:54049)
0812 09:40:48.798777 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.798818 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.800017 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.800659 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.800665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.801011 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.801523 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.801537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.802371 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.802375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.803258 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.803267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.803827 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.803832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.803993 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.804600 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.804615 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:48.807218 (+ 2603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
W20260812 09:40:48.808635 14150 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:35585 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.147.62:35585: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:48.814513 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.809107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:35201)
0812 09:40:48.809220 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.809236 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.809313 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.809604 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.809606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.809736 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.809820 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.809825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.810891 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.810893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.811411 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.811417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.812064 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.812067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.812130 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.812636 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.812657 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:48.814409 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 09:40:48.814900 14007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:48.809205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:35201 (local address 127.3.147.62:35585)
0812 09:40:48.809354 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.809361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.809378 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:40:48.809424 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.809429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.809482 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.809575 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.809934 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.810781 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.811527 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.811950 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.812084 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.814591 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:35201: BlockingRecv error: recv got EOF from 127.3.147.2:35201 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:48.819824 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.815008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:43169)
0812 09:40:48.815170 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.815182 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.815289 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.815537 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.815540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.815689 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.815793 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.815802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.816550 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.816555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.817059 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.817066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.817505 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.817508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.817585 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.818031 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.818044 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:48.819745 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260812 09:40:48.819986 14004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:48.815062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:43169 (local address 127.3.147.62:35585)
0812 09:40:48.815162 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.815167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.815256 (+ 89us) server_negotiation.cc:407] Connection header received
0812 09:40:48.815389 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.815391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.815440 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.815525 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.815894 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.816444 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.817184 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.817397 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.817502 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.819886 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:43169: BlockingRecv error: recv got EOF from 127.3.147.2:43169 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 09:40:48.820080 14150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 09:40:48.916797 14153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:40:48.917039 14153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:40:48.917081 14153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:40:48.921111 14153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:40:48.921171 14153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:40:48.921257 14153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:40:48.926581 14153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/krb5kdc/oryx_127.3.147.3.keytab
--principal=oryx/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:34369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=45805
--tserver_master_addrs=127.3.147.62:35585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:40:48.927603 14153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:40:48.928679 14153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:40:48.938094 14158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.938155 14161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:40:48.938105 14159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:40:48.938514 14153 server_base.cc:1034] running on GCE node
I20260812 09:40:48.938937 14153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:40:48.939651 14153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:40:48.940863 14153 hybrid_clock.cc:648] HybridClock initialized: now 1786527648940837 us; error 41 us; skew 500 ppm
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527648, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:40:48.943813 14153 init.cc:377] Logged in from keytab as oryx/127.3.147.3@KRBTEST.COM (short username oryx)
I20260812 09:40:48.944710 14153 webserver.cc:460] Webserver started at http://127.3.147.3:45805/ using document root <none> and password file <none>
I20260812 09:40:48.945331 14153 fs_manager.cc:359] Metadata directory not provided
I20260812 09:40:48.945457 14153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:40:48.949276 14153 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 09:40:48.951416 14170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.952726 14153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:40:48.952859 14153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "8114690a3b8943118fa5c1485b2af3be"
format_stamp: "Formatted at 2026-08-12 09:40:48 on dist-test-slave-cbbz"
server_key: "849198472bde76741d5a15f659ce6fb1"
server_key_iv: "629867d01beed6083ec877b9fcbad422"
server_key_version: "encryptionkey@0"
I20260812 09:40:48.953410 14153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:40:48.966825 14153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:40:48.970362 14153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:40:48.970562 14153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:40:48.971159 14153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:40:48.971975 14153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:40:48.972046 14153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.972126 14153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:40:48.972201 14153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:40:48.981775 14153 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:34369
I20260812 09:40:48.981822 14300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:34369 every 8 connection(s)
I20260812 09:40:48.982851 14153 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:40:48.988508 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 14153
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 09:40:48.994637 14004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:48.984640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:37993 (local address 127.3.147.62:35585)
0812 09:40:48.984818 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.984824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.985693 (+ 869us) server_negotiation.cc:407] Connection header received
0812 09:40:48.986938 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.986940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.986992 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.987069 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.988224 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.988794 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.990330 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.990541 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.990624 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.994512 (+ 3888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:37993: BlockingRecv error: recv got EOF from 127.3.147.3:37993 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 09:40:48.995253 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.985055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:37993)
0812 09:40:48.985519 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.985562 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.986730 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.987238 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.987245 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.987578 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.988042 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.988055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.988935 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.988943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.990203 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.990216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.990714 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.990718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.990912 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.991495 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.991537 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:48.994399 (+ 2862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
W20260812 09:40:48.995844 14301 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:35585 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.147.62:35585: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
I20260812 09:40:48.999073 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.989522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45222 (local address 127.3.147.62:35585)
0812 09:40:48.989650 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.989658 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.989671 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:48.989803 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.989805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.989839 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.989905 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.990826 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.991453 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.992620 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.992929 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.993046 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.994749 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:40:48.994758 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:40:48.994777 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:40:48.994806 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:40:48.996936 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:48.997371 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:48.997377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:48.997383 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:48.997513 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:40:48.997803 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:40:48.997808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:40:48.997809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:40:48.998149 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:40:48.998355 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 09:40:48.998650 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 09:40:48.998899 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:48 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:49.001612 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:48.996367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:55193)
0812 09:40:48.996457 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:48.996472 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:48.996547 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:48.996766 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:48.996768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:48.996890 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:48.996973 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.996977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.997679 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.997684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.998220 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:48.998227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.998685 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.998689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:48.998788 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:48.999267 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:48.999293 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:49.001520 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 09:40:49.001821 14004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:48.996438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:55193 (local address 127.3.147.62:35585)
0812 09:40:48.996552 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:48.996556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:48.996572 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:40:48.996618 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:48.996621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:48.996666 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:48.996763 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:48.997089 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.997568 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.998345 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:48.998570 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:48.998682 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:49.001687 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:55193: BlockingRecv error: recv got EOF from 127.3.147.3:55193 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 09:40:49 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:49 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:49.007184 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:49.002521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:60913)
0812 09:40:49.002647 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:49.002662 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:49.002735 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:49.002959 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:49.002961 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:49.003078 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:49.003166 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:49.003170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.003882 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.003884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:49.004497 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:49.004505 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.004949 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.004952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:49.005007 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:49.005479 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:49.005491 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:49.007103 (+ 1612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 09:40:49.007444 14301 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 09:40:49.007470 14004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:49.002607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:60913 (local address 127.3.147.62:35585)
0812 09:40:49.002767 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:49.002771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:49.002785 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:49.002820 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:49.002822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:49.002858 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:49.002936 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:49.003271 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.003784 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.004605 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.004843 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.004937 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:49.007307 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:60913: BlockingRecv error: recv got EOF from 127.3.147.3:60913 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 12 09:40:49 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:49 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:49.633351 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:49.627570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:48159)
0812 09:40:49.627894 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:49.627907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:49.628052 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:49.628418 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:49.628422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:49.628677 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:49.628803 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:49.628809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.629607 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.629610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:49.630191 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:49.630199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.630620 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.630623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:49.630687 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:49.631184 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:49.631197 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:49.633220 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":89,"threads_started":1}
W20260812 09:40:49.633504 14306 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:49.627626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:48159 (local address 127.3.147.62:35585)
0812 09:40:49.627919 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:49.627926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:49.628014 (+ 88us) server_negotiation.cc:407] Connection header received
0812 09:40:49.628192 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:49.628195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:49.628254 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:49.628361 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:49.628953 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.629505 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.630296 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.630520 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.630595 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:49.633365 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:48159: BlockingRecv error: recv got EOF from 127.3.147.1:48159 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":92,"threads_started":1}
Aug 12 09:40:49 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:49 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:49.826960 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:49.820686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:33221)
0812 09:40:49.820981 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:49.820995 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:49.821095 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:49.821387 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:49.821389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:49.821583 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:49.821696 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:49.821702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.822493 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.822496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:49.823122 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:49.823132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.823634 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.823637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:49.823709 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:49.824374 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:49.824386 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:49.826825 (+ 2439us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":107,"threads_started":1}
W20260812 09:40:49.827095 14306 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:49.820811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:33221 (local address 127.3.147.62:35585)
0812 09:40:49.820961 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:49.820965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:49.821051 (+ 86us) server_negotiation.cc:407] Connection header received
0812 09:40:49.821185 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:49.821188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:49.821229 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:49.821301 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:49.821830 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.822369 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.823262 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:49.823495 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:49.823587 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:49.826972 (+ 3385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:33221: BlockingRecv error: recv got EOF from 127.3.147.2:33221 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 09:40:50 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:50 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:50.013896 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:50.008043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:58485)
0812 09:40:50.008409 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:50.008424 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:50.008529 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:50.008778 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:50.008780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:50.008955 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:50.009053 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:50.009058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.009768 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.009770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:50.010368 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:50.010375 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.010768 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.010770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:50.010832 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:50.011377 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:50.011389 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:50.013706 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":185,"threads_started":1}
W20260812 09:40:50.013935 14306 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:50.008128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:58485 (local address 127.3.147.62:35585)
0812 09:40:50.008289 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:50.008292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:50.008493 (+ 201us) server_negotiation.cc:407] Connection header received
0812 09:40:50.008609 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:50.008611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:50.008655 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:50.008725 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:50.009169 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.009676 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.010468 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.010673 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.010747 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:50.013826 (+ 3079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:58485: BlockingRecv error: recv got EOF from 127.3.147.3:58485 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 09:40:50 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:50 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:50.640538 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:50.634178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:41319)
0812 09:40:50.634487 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:50.634508 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:50.634604 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:50.634870 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:50.634872 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:50.635063 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:50.635186 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:50.635193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.635923 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.635926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:50.636539 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:50.636548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.636986 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.636989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:50.637059 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:50.637604 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:50.637616 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:50.640235 (+ 2619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":96,"threads_started":1}
W20260812 09:40:50.640576 14312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:50.634259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:41319 (local address 127.3.147.62:35585)
0812 09:40:50.634560 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:50.634564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:50.634578 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:50.634701 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:50.634703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:50.634747 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:50.634826 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:50.635287 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.635817 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.636656 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.636878 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.636953 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:50.640400 (+ 3447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:41319: BlockingRecv error: recv got EOF from 127.3.147.1:41319 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":102,"threads_started":1}
Aug 12 09:40:50 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:50 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:50.833552 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:50.827798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:40725)
0812 09:40:50.828177 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:50.828194 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:50.828345 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:50.828645 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:50.828649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:50.828882 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:50.828984 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:50.828990 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.829781 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.829786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:50.830343 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:50.830350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.830809 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.830812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:50.830900 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:50.831428 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:50.831440 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:50.833351 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":96,"threads_started":1}
W20260812 09:40:50.833572 14312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:50.827944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:40725 (local address 127.3.147.62:35585)
0812 09:40:50.828121 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:50.828126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:50.828280 (+ 154us) server_negotiation.cc:407] Connection header received
0812 09:40:50.828457 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:50.828460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:50.828513 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:50.828603 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:50.829109 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.829662 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.830472 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:50.830693 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:50.830803 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:50.833473 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:40725: BlockingRecv error: recv got EOF from 127.3.147.2:40725 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 09:40:51 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:51 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:51.025848 14314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:51.014706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:57593)
0812 09:40:51.015055 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:51.015074 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:51.015228 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:51.015515 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:51.015519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:51.015733 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:51.015827 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:51.015833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.016714 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.016719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:51.017282 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:51.017289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.017751 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.017755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:51.017852 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:51.023483 (+ 5631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:51.023496 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:51.025645 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
W20260812 09:40:51.025887 14312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:51.014887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:57593 (local address 127.3.147.62:35585)
0812 09:40:51.015056 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:51.015062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:51.015162 (+ 100us) server_negotiation.cc:407] Connection header received
0812 09:40:51.015331 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:51.015335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:51.015391 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:51.015495 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:51.015947 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.016580 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.017404 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.017625 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.017736 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:51.025779 (+ 8043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:57593: BlockingRecv error: recv got EOF from 127.3.147.3:57593 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 09:40:51 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:51 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:51.647053 14316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:51.641425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:40821)
0812 09:40:51.641741 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:51.641756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:51.641866 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:51.642171 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:51.642174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:51.642368 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:51.642470 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:51.642475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.643262 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.643266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:51.643834 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:51.643842 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.644300 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.644303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:51.644372 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:51.644897 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:51.644909 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:51.646913 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":90,"threads_started":1}
W20260812 09:40:51.647197 14317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:51.641543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:40821 (local address 127.3.147.62:35585)
0812 09:40:51.641821 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:51.641825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:51.641839 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:51.641955 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:51.641957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:51.642031 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:51.642119 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:51.642595 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.643136 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.643946 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.644200 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.644281 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:51.647040 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:40821: BlockingRecv error: recv got EOF from 127.3.147.1:40821 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":64,"threads_started":1}
Aug 12 09:40:51 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:51 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:51.840747 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:51.834459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:40753)
0812 09:40:51.834741 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:51.834780 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:51.834909 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:51.835247 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:51.835252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:51.835516 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:51.835645 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:51.835654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.836742 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.836745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:51.837307 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:51.837315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.837791 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.837794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:51.837859 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:51.838384 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:51.838395 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:51.840571 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":104,"threads_started":1}
W20260812 09:40:51.840840 14317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:51.834539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:40753 (local address 127.3.147.62:35585)
0812 09:40:51.834739 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:51.834743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:51.834866 (+ 123us) server_negotiation.cc:407] Connection header received
0812 09:40:51.835033 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:51.835037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:51.835103 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:51.835210 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:51.835801 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.836634 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.837437 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:51.837667 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:51.837771 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:51.840711 (+ 2940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:40753: BlockingRecv error: recv got EOF from 127.3.147.2:40753 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 12 09:40:52 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:52 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:52.032214 14320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:52.026653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:41277)
0812 09:40:52.026919 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:52.026933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:52.027032 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:52.027292 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:52.027294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:52.027472 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:52.027573 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:52.027577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.028365 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.028367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:52.028914 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:52.028922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.029349 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.029352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:52.029416 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:52.029878 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:52.029890 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:52.032016 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
W20260812 09:40:52.032287 14317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:52.026758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:41277 (local address 127.3.147.62:35585)
0812 09:40:52.026902 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:52.026905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:52.026987 (+ 82us) server_negotiation.cc:407] Connection header received
0812 09:40:52.027100 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:52.027102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:52.027182 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:52.027251 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:52.027689 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.028266 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.029014 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.029240 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.029320 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:52.032195 (+ 2875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:41277: BlockingRecv error: recv got EOF from 127.3.147.3:41277 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 09:40:52 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:52 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:52.653731 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:52.647848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:38867)
0812 09:40:52.648187 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:52.648204 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:52.648321 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:52.648782 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:52.648784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:52.648967 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:52.649072 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:52.649077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.649842 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.649845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:52.650410 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:52.650417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.650839 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.650841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:52.650903 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:52.651353 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:52.651365 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:52.653612 (+ 2247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":91,"threads_started":1}
W20260812 09:40:52.653873 14322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:52.647884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:38867 (local address 127.3.147.62:35585)
0812 09:40:52.648553 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:52.648557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:52.648569 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:40:52.648608 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:52.648610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:52.648649 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:52.648725 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:52.649208 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.649743 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.650515 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.650740 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.650815 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:52.653746 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:38867: BlockingRecv error: recv got EOF from 127.3.147.1:38867 (error 108)
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":70,"threads_started":1}
Aug 12 09:40:52 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:52 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:52.847103 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:52.841559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:45899)
0812 09:40:52.841920 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:52.841935 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:52.842074 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:52.842364 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:52.842367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:52.842591 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:52.842693 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:52.842698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.843455 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.843461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:52.844043 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:52.844051 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.844546 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.844550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:52.844637 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:52.845145 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:52.845157 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:52.846984 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":88,"threads_started":1}
W20260812 09:40:52.847308 14322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:52.841660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:45899 (local address 127.3.147.62:35585)
0812 09:40:52.841852 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:52.841858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:52.842020 (+ 162us) server_negotiation.cc:407] Connection header received
0812 09:40:52.842192 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:52.842196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:52.842248 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:52.842338 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:52.842813 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.843344 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.844203 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:52.844421 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:52.844542 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:52.847178 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:45899: BlockingRecv error: recv got EOF from 127.3.147.2:45899 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 09:40:53 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:53 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:53.039019 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:53.033050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:44669)
0812 09:40:53.033422 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:53.033435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:53.033582 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:53.033870 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:53.033873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:53.034090 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:53.034201 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:53.034207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.035034 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.035038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:53.035934 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:53.035947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.036459 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.036461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:53.036522 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:53.037078 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:53.037092 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:53.038904 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":130,"threads_started":1}
W20260812 09:40:53.039158 14322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:53.033167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:44669 (local address 127.3.147.62:35585)
0812 09:40:53.033359 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:53.033365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:53.033520 (+ 155us) server_negotiation.cc:407] Connection header received
0812 09:40:53.033683 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:53.033687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:53.033757 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:53.033847 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:53.034333 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.034890 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.036065 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.036360 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.036440 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:53.039038 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:44669: BlockingRecv error: recv got EOF from 127.3.147.3:44669 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 12 09:40:53 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:53 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:53.660697 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:53.654630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:33385)
0812 09:40:53.654959 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:53.654973 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:53.655063 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:53.655353 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:53.655355 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:53.655550 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:53.655653 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:53.655659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.656805 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.656809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:53.657528 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:53.657539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.658029 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.658032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:53.658108 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:53.658617 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:53.658629 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:53.660568 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":94,"threads_started":1}
W20260812 09:40:53.660844 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:53.654766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:33385 (local address 127.3.147.62:35585)
0812 09:40:53.655044 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:53.655047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:53.655061 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:53.655183 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:53.655185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:53.655229 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:53.655307 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:53.655792 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.656647 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.657679 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.657894 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.658021 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:53.660711 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:33385: BlockingRecv error: recv got EOF from 127.3.147.1:33385 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":124,"threads_started":1}
Aug 12 09:40:53 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:53 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:53.854071 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:53.847941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:56557)
0812 09:40:53.848297 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:53.848326 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:53.848441 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:53.848729 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:53.848733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:53.848959 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:53.849055 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:53.849061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.849855 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.849859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:53.850459 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:53.850467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.850951 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.850955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:53.851055 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:53.851631 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:53.851649 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:53.853945 (+ 2296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":109,"threads_started":1}
W20260812 09:40:53.854267 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:53.848119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:56557 (local address 127.3.147.62:35585)
0812 09:40:53.848292 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:53.848297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:53.848387 (+ 90us) server_negotiation.cc:407] Connection header received
0812 09:40:53.848550 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:53.848553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:53.848608 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:53.848699 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:53.849180 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.849729 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.850582 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:53.850829 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:53.850944 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:53.854099 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:56557: BlockingRecv error: recv got EOF from 127.3.147.2:56557 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 09:40:54 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:54 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:54.045866 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:54.039894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:49619)
0812 09:40:54.040231 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:54.040245 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:54.040377 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:54.040712 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:54.040714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:54.040957 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:54.041053 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:54.041058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.041985 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.041989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:54.042579 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:54.042588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.043061 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.043064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:54.043162 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:54.043655 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:54.043667 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:54.045750 (+ 2083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":104,"threads_started":1}
W20260812 09:40:54.046003 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:54.039920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:49619 (local address 127.3.147.62:35585)
0812 09:40:54.040080 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:54.040091 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:54.040329 (+ 238us) server_negotiation.cc:407] Connection header received
0812 09:40:54.040480 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:54.040484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:54.040540 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:54.040664 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:54.041178 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.041874 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.042703 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.042942 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.043056 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:54.045886 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:49619: BlockingRecv error: recv got EOF from 127.3.147.3:49619 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 09:40:54 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:54 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:54.667212 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:54.661498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:45929)
0812 09:40:54.661789 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:54.661812 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:54.661907 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:54.662194 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:54.662197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:54.662389 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:54.662517 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:54.662523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.663498 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.663501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:54.664094 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:54.664102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.664597 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.664600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:54.664673 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:54.665218 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:54.665230 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:54.667065 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":75,"threads_started":1}
W20260812 09:40:54.667405 14331 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:54.661603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:45929 (local address 127.3.147.62:35585)
0812 09:40:54.661874 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:54.661877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:54.661892 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:40:54.661990 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:54.661993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:54.662036 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:54.662123 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:54.662707 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.663384 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.664252 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.664489 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.664579 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:54.667249 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:45929: BlockingRecv error: recv got EOF from 127.3.147.1:45929 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
Aug 12 09:40:54 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:54 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:54.860663 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:54.854908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:33625)
0812 09:40:54.855233 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:54.855248 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:54.855359 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:54.855602 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:54.855604 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:54.855798 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:54.855900 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:54.855905 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.856745 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.856749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:54.857414 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:54.857423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.857886 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.857889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:54.857960 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:54.858569 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:54.858581 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:54.860539 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":94,"threads_started":1}
W20260812 09:40:54.860848 14331 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:54.855036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:33625 (local address 127.3.147.62:35585)
0812 09:40:54.855227 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:54.855232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:54.855303 (+ 71us) server_negotiation.cc:407] Connection header received
0812 09:40:54.855443 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:54.855445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:54.855490 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:54.855556 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:54.855999 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.856627 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.857523 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:54.857753 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:54.857845 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:54.860685 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:33625: BlockingRecv error: recv got EOF from 127.3.147.2:33625 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 12 09:40:55 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:55 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:55.053040 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:55.046789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:50971)
0812 09:40:55.047076 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:55.047097 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:55.047204 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:55.047456 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:55.047459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:55.047678 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:55.047778 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:55.047783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.048612 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.048615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:55.049185 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:55.049193 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.049729 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.049733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:55.049836 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:55.050534 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:55.050547 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:55.052914 (+ 2367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":89,"threads_started":1}
W20260812 09:40:55.053325 14331 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:55.046854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:50971 (local address 127.3.147.62:35585)
0812 09:40:55.047001 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:55.047005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:55.047167 (+ 162us) server_negotiation.cc:407] Connection header received
0812 09:40:55.047296 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:55.047299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:55.047348 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:55.047414 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:55.047884 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.048498 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.049306 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.049547 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.049646 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:55.053044 (+ 3398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:50971: BlockingRecv error: recv got EOF from 127.3.147.3:50971 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 09:40:55 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:55 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:55.673974 14334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:55.668003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:41623)
0812 09:40:55.668432 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:55.668447 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:55.668547 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:55.668807 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:55.668809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:55.669088 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:55.669215 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:55.669222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.669988 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.669991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:55.670646 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:55.670658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.671280 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.671287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:55.671406 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:55.671952 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:55.671964 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:55.673845 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":147,"threads_started":1}
W20260812 09:40:55.674110 14336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:55.668118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:41623 (local address 127.3.147.62:35585)
0812 09:40:55.668484 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:55.668488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:55.668501 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:55.668635 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:55.668638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:55.668682 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:55.668771 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:55.669320 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.669871 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.670768 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.671124 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.671245 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:55.673974 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:41623: BlockingRecv error: recv got EOF from 127.3.147.1:41623 (error 108)
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":153,"threads_started":1}
Aug 12 09:40:55 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:55 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:55.867225 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:55.861426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:40829)
0812 09:40:55.861793 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:55.861817 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:55.861922 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:55.862164 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:55.862166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:55.862354 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:55.862476 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:55.862482 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.863365 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.863369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:55.863989 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:55.863999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.864508 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.864510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:55.864579 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:55.865073 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:55.865086 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:55.867089 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":108,"threads_started":1}
W20260812 09:40:55.867410 14336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:55.861514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:40829 (local address 127.3.147.62:35585)
0812 09:40:55.861655 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:55.861659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:55.861886 (+ 227us) server_negotiation.cc:407] Connection header received
0812 09:40:55.861988 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:55.861991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:55.862035 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:55.862129 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:55.862577 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.863248 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.864116 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:55.864405 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:55.864487 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:55.867269 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:40829: BlockingRecv error: recv got EOF from 127.3.147.2:40829 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 09:40:56 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:56 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:56.065766 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:56.053837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:52161)
0812 09:40:56.054247 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:56.054262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:56.054409 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:56.054725 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:56.054729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:56.054983 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:56.055089 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:56.055097 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.055904 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.055909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:56.056579 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:56.056589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.057098 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.057102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:56.057201 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:56.062855 (+ 5654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:56.062870 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:56.065644 (+ 2774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":94,"threads_started":1}
W20260812 09:40:56.066030 14336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:56.053948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:52161 (local address 127.3.147.62:35585)
0812 09:40:56.054150 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:56.054156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:56.054347 (+ 191us) server_negotiation.cc:407] Connection header received
0812 09:40:56.054529 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:56.054534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:56.054596 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:56.054699 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:56.055229 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.055776 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.056714 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.056955 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.057091 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:56.065789 (+ 8698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:52161: BlockingRecv error: recv got EOF from 127.3.147.3:52161 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 09:40:56 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:56 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:56.680687 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:56.674839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:53869)
0812 09:40:56.675115 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:56.675130 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:56.675226 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:56.675563 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:56.675566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:56.675770 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:56.675874 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:56.675880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.676835 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.676838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:56.677410 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:56.677419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.677926 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.677929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:56.678002 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:56.678531 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:56.678543 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:56.680568 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":93,"threads_started":1}
W20260812 09:40:56.680904 14340 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:56.675039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:53869 (local address 127.3.147.62:35585)
0812 09:40:56.675285 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:56.675288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:56.675302 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:40:56.675339 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:56.675341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:56.675383 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:56.675491 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:56.675995 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.676704 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.677521 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.677800 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.677894 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:56.680715 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:53869: BlockingRecv error: recv got EOF from 127.3.147.1:53869 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":84,"threads_started":1}
Aug 12 09:40:56 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:56 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:56.874409 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:56.868187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:33137)
0812 09:40:56.868577 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:56.868596 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:56.868745 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:56.869216 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:56.869220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:56.869434 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:56.869543 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:56.869548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.870489 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.870492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:56.871127 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:56.871136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.871594 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.871597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:56.871674 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:56.872310 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:56.872324 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:56.874283 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":97,"threads_started":1}
W20260812 09:40:56.874583 14340 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:56.868231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:33137 (local address 127.3.147.62:35585)
0812 09:40:56.868399 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:56.868404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:56.868694 (+ 290us) server_negotiation.cc:407] Connection header received
0812 09:40:56.868844 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:56.868850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:56.868921 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:56.869006 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:56.869655 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.870382 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.871246 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:56.871488 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:56.871577 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:56.874425 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:33137: BlockingRecv error: recv got EOF from 127.3.147.2:33137 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 09:40:57 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:57 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:57.072474 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:57.066518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:51419)
0812 09:40:57.066819 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:57.066836 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:57.066952 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:57.067295 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:57.067299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:57.067500 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:57.067608 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:57.067614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.068579 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.068582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:57.069169 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:57.069178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.069619 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.069621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:57.069683 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:57.070190 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:57.070202 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:57.072358 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":120,"threads_started":1}
W20260812 09:40:57.072620 14340 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:57.066569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:51419 (local address 127.3.147.62:35585)
0812 09:40:57.066714 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:57.066718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:57.066926 (+ 208us) server_negotiation.cc:407] Connection header received
0812 09:40:57.067033 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:57.067036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:57.067125 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:57.067201 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:57.067729 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.068459 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.069282 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.069515 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.069606 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:57.072505 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:51419: BlockingRecv error: recv got EOF from 127.3.147.3:51419 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 09:40:57 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:57 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:57.686846 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:57.681479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:40037)
0812 09:40:57.681758 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:57.681771 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:57.681872 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:57.682127 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:57.682129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:57.682323 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:57.682423 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:57.682428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.683155 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.683157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:57.683710 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:57.683718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.684225 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.684228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:57.684293 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:57.684878 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:57.684890 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:57.686728 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":106,"threads_started":1}
W20260812 09:40:57.687047 14346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:57.681570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:40037 (local address 127.3.147.62:35585)
0812 09:40:57.681813 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:57.681816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:57.681829 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:40:57.681951 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:57.681954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:57.681998 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:57.682079 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:57.682522 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.683046 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.683823 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.684061 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.684224 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:57.686868 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:40037: BlockingRecv error: recv got EOF from 127.3.147.1:40037 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
Aug 12 09:40:57 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:57 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:57.880900 14347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:57.875234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:43279)
0812 09:40:57.875539 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:57.875554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:57.875669 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:57.875929 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:57.875931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:57.876128 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:57.876260 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:57.876265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.877112 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.877115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:57.877717 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:57.877726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.878188 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.878191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:57.878252 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:57.878830 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:57.878842 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:57.880784 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":100,"threads_started":1}
W20260812 09:40:57.881057 14346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:57.875288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:43279 (local address 127.3.147.62:35585)
0812 09:40:57.875441 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:57.875446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:57.875640 (+ 194us) server_negotiation.cc:407] Connection header received
0812 09:40:57.875749 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:57.875751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:57.875797 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:57.875867 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:57.876366 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.876984 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.877816 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:57.878082 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:57.878172 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:57.880906 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:43279: BlockingRecv error: recv got EOF from 127.3.147.2:43279 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 09:40:58 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:58 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:58.079116 14348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:58.073321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:47251)
0812 09:40:58.073665 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:58.073682 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:58.073819 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:58.074082 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:58.074085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:58.074305 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:58.074398 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:58.074403 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.075180 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.075183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:58.075739 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:58.075747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.076253 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.076257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:58.076346 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:58.076931 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:58.076944 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:58.078996 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
W20260812 09:40:58.079324 14346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:58.073495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:47251 (local address 127.3.147.62:35585)
0812 09:40:58.073664 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:58.073668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:58.073759 (+ 91us) server_negotiation.cc:407] Connection header received
0812 09:40:58.073914 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:58.073919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:58.073972 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:58.074054 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:58.074512 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.075054 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.075854 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.076101 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.076250 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:58.079183 (+ 2933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:47251: BlockingRecv error: recv got EOF from 127.3.147.3:47251 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 09:40:58 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:58 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:58.693434 14349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:58.687695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:38421)
0812 09:40:58.687985 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:58.687999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:58.688076 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:58.688400 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:58.688402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:58.688602 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:58.688701 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:58.688706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.689519 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.689522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:58.690088 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:58.690097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.690571 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.690574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:58.690654 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:58.691190 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:58.691201 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:58.693242 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":110,"threads_started":1}
W20260812 09:40:58.693593 14350 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:58.687727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:38421 (local address 127.3.147.62:35585)
0812 09:40:58.688017 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:58.688021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:58.688056 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:40:58.688228 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:58.688231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:58.688279 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:58.688362 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:58.688801 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.689409 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.690215 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.690474 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.690554 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:58.693413 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:38421: BlockingRecv error: recv got EOF from 127.3.147.1:38421 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":122,"threads_started":1}
Aug 12 09:40:58 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:58 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:58.886889 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:58.881717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:43509)
0812 09:40:58.882003 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:58.882017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:58.882112 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:58.882348 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:58.882351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:58.882540 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:58.882642 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:58.882647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.883408 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.883410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:58.884009 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:58.884017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.884458 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.884461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:58.884521 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:58.885013 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:58.885024 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:58.886748 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":101,"threads_started":1}
W20260812 09:40:58.886968 14350 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:58.881777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:43509 (local address 127.3.147.62:35585)
0812 09:40:58.881928 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:58.881931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:58.882085 (+ 154us) server_negotiation.cc:407] Connection header received
0812 09:40:58.882192 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:58.882194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:58.882237 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:58.882311 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:58.882769 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.883307 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.884112 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:58.884354 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:58.884429 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:58.886876 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:43509: BlockingRecv error: recv got EOF from 127.3.147.2:43509 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 09:40:59 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:59 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:59.085696 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:59.080050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:37737)
0812 09:40:59.080380 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:59.080394 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:59.080488 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:59.080735 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:59.080737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:59.080940 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:59.081040 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:59.081045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.081776 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.081779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:59.082367 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:59.082376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.082777 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.082780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:59.082848 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:59.083348 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:59.083360 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:59.085547 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":146,"threads_started":1}
W20260812 09:40:59.085791 14350 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:59.080219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:37737 (local address 127.3.147.62:35585)
0812 09:40:59.080352 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:59.080356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:59.080517 (+ 161us) server_negotiation.cc:407] Connection header received
0812 09:40:59.080562 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:59.080564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:59.080622 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:59.080701 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:59.081145 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.081676 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.082463 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.082671 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.082744 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:59.085672 (+ 2928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:37737: BlockingRecv error: recv got EOF from 127.3.147.3:37737 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 09:40:59 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:59 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:59.700487 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:59.694362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:44431)
0812 09:40:59.694718 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:59.694733 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:59.694840 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:59.695165 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:59.695167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:59.695380 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:59.695486 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:59.695493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.696337 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.696341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:59.696984 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:59.697000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.697501 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.697504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:59.697594 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:59.698206 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:59.698219 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:59.700261 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":95,"threads_started":1}
W20260812 09:40:59.700531 14354 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:59.694532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:44431 (local address 127.3.147.62:35585)
0812 09:40:59.694777 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:59.694781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:59.694798 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:40:59.694943 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:59.694945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:59.694991 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:59.695076 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:59.695610 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.696224 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.697137 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.697375 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.697494 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:59.700396 (+ 2902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:44431: BlockingRecv error: recv got EOF from 127.3.147.1:44431 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":68,"threads_started":1}
Aug 12 09:40:59 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:40:59 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:40:59.893625 14356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:40:59.887867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:49013)
0812 09:40:59.888192 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 09:40:59.888205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:40:59.888314 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:40:59.888584 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:40:59.888587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:40:59.888781 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:40:59.888883 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:59.888888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.889655 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.889658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:59.890279 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:40:59.890289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.890741 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.890744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:40:59.890816 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:59.891457 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:40:59.891471 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:40:59.893458 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
W20260812 09:40:59.893797 14354 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:40:59.887910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:49013 (local address 127.3.147.62:35585)
0812 09:40:59.888058 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 09:40:59.888062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:40:59.888272 (+ 210us) server_negotiation.cc:407] Connection header received
0812 09:40:59.888382 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:40:59.888384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:40:59.888435 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:40:59.888521 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:40:59.888991 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.889528 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.890404 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:40:59.890629 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:40:59.890710 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:40:59.893674 (+ 2964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:49013: BlockingRecv error: recv got EOF from 127.3.147.2:49013 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 09:41:00 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:00 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:00.092473 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:00.086497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:55625)
0812 09:41:00.086845 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:00.086859 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:00.086994 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:00.087265 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:00.087268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:00.087485 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:00.087578 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:00.087584 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.088421 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.088426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:00.088983 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:00.088990 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.089480 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.089483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:00.089570 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:00.090183 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:00.090199 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:00.092359 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":108,"threads_started":1}
W20260812 09:41:00.092711 14354 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:00.086598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:55625 (local address 127.3.147.62:35585)
0812 09:41:00.086792 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:00.086797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:00.086951 (+ 154us) server_negotiation.cc:407] Connection header received
0812 09:41:00.087098 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:00.087102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:00.087157 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:00.087240 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:00.087697 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.088316 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.089135 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.089365 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.089474 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:00.092600 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:55625: BlockingRecv error: recv got EOF from 127.3.147.3:55625 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 12 09:41:00 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:00 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:00.706866 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:00.701299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:56607)
0812 09:41:00.701597 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:00.701611 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:00.701719 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:00.701991 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:00.701993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:00.702194 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:00.702317 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:00.702323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.703109 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.703112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:00.703687 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:00.703695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.704125 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.704128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:00.704260 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:00.704766 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:00.704778 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:00.706739 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":84,"threads_started":1}
W20260812 09:41:00.707093 14359 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:00.701427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:56607 (local address 127.3.147.62:35585)
0812 09:41:00.701669 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:00.701672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:00.701687 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:41:00.701789 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:00.701791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:00.701844 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:00.701919 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:00.702438 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.702994 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.703786 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.704017 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.704094 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:00.706933 (+ 2839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:56607: BlockingRecv error: recv got EOF from 127.3.147.1:56607 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
Aug 12 09:41:00 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:00 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:00.900130 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:00.894472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:55533)
0812 09:41:00.894818 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:00.894831 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:00.894978 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:00.895273 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:00.895276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:00.895484 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:00.895581 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:00.895612 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.896460 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.896465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:00.897031 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:00.897039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.897505 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.897509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:00.897598 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:00.898135 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:00.898147 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:00.900010 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":100,"threads_started":1}
W20260812 09:41:00.900389 14359 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:00.894565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:55533 (local address 127.3.147.62:35585)
0812 09:41:00.894759 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:00.894764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:00.894915 (+ 151us) server_negotiation.cc:407] Connection header received
0812 09:41:00.895091 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:00.895095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:00.895158 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:00.895244 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:00.895728 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.896351 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.897161 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:00.897390 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:00.897500 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:00.900192 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:55533: BlockingRecv error: recv got EOF from 127.3.147.2:55533 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 12 09:41:01 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:01 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:01.101465 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:01.093268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:54025)
0812 09:41:01.093554 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:01.093568 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:01.093662 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:01.093890 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:01.093892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:01.094069 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:01.094185 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:01.094190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.094908 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.094910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:01.095605 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:01.095617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.096193 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.096200 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:01.096287 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:01.099531 (+ 3244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:01.099544 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:01.101348 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":104,"threads_started":1}
W20260812 09:41:01.101598 14359 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:01.093322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:54025 (local address 127.3.147.62:35585)
0812 09:41:01.093598 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:01.093602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:01.093615 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:41:01.093743 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:01.093745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:01.093789 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:01.093858 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:01.094281 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.094799 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.095726 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.096033 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.096174 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:01.101483 (+ 5309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:54025: BlockingRecv error: recv got EOF from 127.3.147.3:54025 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Aug 12 09:41:01 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:01 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:01.713543 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:01.707803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:54811)
0812 09:41:01.708076 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:01.708096 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:01.708226 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:01.708603 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:01.708606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:01.708797 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:01.708900 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:01.708906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.709715 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.709719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:01.710372 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:01.710383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.710832 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.710835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:01.710907 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:01.711441 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:01.711455 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:01.713392 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
W20260812 09:41:01.713837 14363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:01.707846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:54811 (local address 127.3.147.62:35585)
0812 09:41:01.708084 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:01.708087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:01.708178 (+ 91us) server_negotiation.cc:407] Connection header received
0812 09:41:01.708338 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:01.708341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:01.708441 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:01.708551 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:01.709015 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.709589 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.710491 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.710723 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.710803 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:01.713543 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:54811: BlockingRecv error: recv got EOF from 127.3.147.1:54811 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
Aug 12 09:41:01 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:01 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:01.906988 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:01.901021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:38413)
0812 09:41:01.901387 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:01.901405 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:01.901563 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:01.901853 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:01.901856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:01.902078 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:01.902176 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:01.902182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.902972 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.902978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:01.903567 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:01.903575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.904043 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.904047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:01.904166 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:01.904720 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:01.904733 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:01.906810 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":122,"threads_started":1}
W20260812 09:41:01.907043 14363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:01.901219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:38413 (local address 127.3.147.62:35585)
0812 09:41:01.901388 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:01.901394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:01.901488 (+ 94us) server_negotiation.cc:407] Connection header received
0812 09:41:01.901682 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:01.901685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:01.901738 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:01.901830 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:01.902298 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.902843 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.903691 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:01.903931 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:01.904037 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:01.906936 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:38413: BlockingRecv error: recv got EOF from 127.3.147.2:38413 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 09:41:02 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:02 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:02.108095 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:02.102300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:34323)
0812 09:41:02.102620 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:02.102633 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:02.102772 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:02.103051 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:02.103054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:02.103275 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:02.103370 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:02.103376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.104131 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.104166 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:02.104736 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:02.104745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.105224 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.105228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:02.105318 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:02.105899 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:02.105914 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:02.107968 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":91,"threads_started":1}
W20260812 09:41:02.108273 14363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:02.102366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:34323 (local address 127.3.147.62:35585)
0812 09:41:02.102540 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:02.102545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:02.102710 (+ 165us) server_negotiation.cc:407] Connection header received
0812 09:41:02.102877 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:02.102880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:02.102934 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:02.103024 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:02.103490 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.104015 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.104867 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.105093 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.105220 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:02.108118 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:34323: BlockingRecv error: recv got EOF from 127.3.147.3:34323 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 09:41:02 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:02 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:02.720563 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:02.714477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:37463)
0812 09:41:02.714786 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:02.714800 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:02.714912 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:02.715225 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:02.715228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:02.715487 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:02.715639 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:02.715646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.716673 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.716678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:02.717267 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:02.717275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.717764 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.717767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:02.717859 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:02.718426 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:02.718438 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:02.720433 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
W20260812 09:41:02.720742 14368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:02.714569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:37463 (local address 127.3.147.62:35585)
0812 09:41:02.714974 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:02.714980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:02.714997 (+ 17us) server_negotiation.cc:407] Connection header received
0812 09:41:02.715046 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:02.715048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:02.715090 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:02.715181 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:02.715764 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.716517 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.717389 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.717644 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.717757 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:02.720569 (+ 2812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:37463: BlockingRecv error: recv got EOF from 127.3.147.1:37463 (error 108)
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":118,"threads_started":1}
Aug 12 09:41:02 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:02 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:02.913841 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:02.907853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:37295)
0812 09:41:02.908221 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:02.908235 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:02.908347 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:02.908573 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:02.908575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:02.908765 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:02.908860 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:02.908865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.909630 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.909633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:02.910220 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:02.910228 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.910662 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.910665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:02.910745 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:02.911355 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:02.911366 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:02.913648 (+ 2282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":73,"threads_started":1}
W20260812 09:41:02.913904 14368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:02.907990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:37295 (local address 127.3.147.62:35585)
0812 09:41:02.908191 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:02.908195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:02.908306 (+ 111us) server_negotiation.cc:407] Connection header received
0812 09:41:02.908423 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:02.908425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:02.908468 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:02.908536 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:02.908965 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.909522 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.910322 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:02.910554 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:02.910634 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:02.913779 (+ 3145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:37295: BlockingRecv error: recv got EOF from 127.3.147.2:37295 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 09:41:03 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:03 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.3@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:03.115195 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:03.108957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.3:51819)
0812 09:41:03.109263 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:03.109288 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:03.109425 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:03.109715 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:03.109718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:03.109951 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:03.110159 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:03.110169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.111315 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.111321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:03.111962 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:03.111972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.112448 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.112451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:03.112521 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:03.113020 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:03.113033 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:03.115060 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
W20260812 09:41:03.115305 14368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:03.109057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:51819 (local address 127.3.147.62:35585)
0812 09:41:03.109236 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:03.109239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:03.109410 (+ 171us) server_negotiation.cc:407] Connection header received
0812 09:41:03.109507 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:03.109510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:03.109573 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:03.109652 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:03.110297 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.111117 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.112084 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.112325 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.112406 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:03.115191 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.3:51819: BlockingRecv error: recv got EOF from 127.3.147.3:51819 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 09:41:03 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:03 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.1@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:03.726755 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:03.721474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.1:57493)
0812 09:41:03.721756 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:03.721770 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:03.721877 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:03.722122 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:03.722124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:03.722313 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:03.722411 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:03.722417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.723132 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.723134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:03.723691 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:03.723698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.724184 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.724190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:03.724266 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:03.724821 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:03.724833 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:03.726630 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":93,"threads_started":1}
W20260812 09:41:03.726955 14372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:03.721521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:57493 (local address 127.3.147.62:35585)
0812 09:41:03.721765 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:03.721769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:03.721846 (+ 77us) server_negotiation.cc:407] Connection header received
0812 09:41:03.721958 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:03.721960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:03.722004 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:03.722084 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:03.722511 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.723036 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.723796 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.724035 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.724115 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:03.726799 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.1:57493: BlockingRecv error: recv got EOF from 127.3.147.1:57493 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
Aug 12 09:41:03 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 09:41:03 dist-test-slave-cbbz krb5kdc[13151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.147.2@KRBTEST.COM for oryx/127.3.147.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 09:41:03.919974 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:03.914714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35585 (local address 127.3.147.2:50325)
0812 09:41:03.914972 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:03.914984 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:03.915075 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:03.915315 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:03.915317 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:03.915512 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:03.915610 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:03.915616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.916417 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.916420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:03.916998 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:03.917006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.917394 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.917397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:03.917456 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:03.917955 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:03.917966 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:03.919852 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.147.62:35585: Server oryx/127.3.147.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
W20260812 09:41:03.920133 14372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:03.914795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:50325 (local address 127.3.147.62:35585)
0812 09:41:03.914942 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:03.914945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:03.915038 (+ 93us) server_negotiation.cc:407] Connection header received
0812 09:41:03.915163 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:03.915165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:03.915209 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:03.915280 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:03.915706 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.916311 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.917093 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:03.917302 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:03.917377 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:03.919984 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.147.2:50325: BlockingRecv error: recv got EOF from 127.3.147.2:50325 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:41:03.992195 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 13831
I20260812 09:41:04.001874 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 14006
I20260812 09:41:04.008762 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 14153
I20260812 09:41:04.016348 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 13751
2026-08-12T09:41:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18076 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T09:41:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:41:04Z Disabled control of system clock
I20260812 09:41:04.049832 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:40889
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:40889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 09:41:04.162850 14377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:04.163111 14377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:04.163146 14377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:04.167260 14377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:41:04.167323 14377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:04.167338 14377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:04.167356 14377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:41:04.167371 14377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:41:04.172160 14377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:40889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:40889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:04.173283 14377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:04.174713 14377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:41:04.183856 14377 server_base.cc:1034] running on GCE node
W20260812 09:41:04.183969 14386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:04.183982 14382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:04.183981 14383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:04.184926 14377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:04.186293 14377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:04.187477 14377 hybrid_clock.cc:648] HybridClock initialized: now 1786527664187453 us; error 42 us; skew 500 ppm
I20260812 09:41:04.189389 14377 webserver.cc:460] Webserver started at http://127.3.147.62:43701/ using document root <none> and password file <none>
I20260812 09:41:04.190032 14377 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:04.190124 14377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:04.190359 14377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:04.192456 14377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "a8708e7efc6b46d7a2c95dc138918d15"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "571485f6e25c5770d3ee362dcacbdba9"
server_key_iv: "ade9a2e40ae91fb8487bd80ea867060e"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.193011 14377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "a8708e7efc6b46d7a2c95dc138918d15"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "571485f6e25c5770d3ee362dcacbdba9"
server_key_iv: "ade9a2e40ae91fb8487bd80ea867060e"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.196879 14377 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260812 09:41:04.199563 14392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.201025 14377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:41:04.201193 14377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "a8708e7efc6b46d7a2c95dc138918d15"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "571485f6e25c5770d3ee362dcacbdba9"
server_key_iv: "ade9a2e40ae91fb8487bd80ea867060e"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.201309 14377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:04.234687 14377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:04.235440 14377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:04.235643 14377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:04.243011 14377 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:40889
I20260812 09:41:04.243023 14452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:40889 every 8 connection(s)
I20260812 09:41:04.243993 14377 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:41:04.246137 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 14377
I20260812 09:41:04.246284 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:41:04.246572 3660 external_mini_cluster.cc:1428] Setting key 7d3eafdcc8767d5af9c41c07e0e1f183
I20260812 09:41:04.247440 14453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:04.254098 14453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15: Bootstrap starting.
I20260812 09:41:04.255878 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:04.248009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.3.147.62:40889)
0812 09:41:04.248691 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:04.248710 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:04.248746 (+ 36us) server_negotiation.cc:407] Connection header received
0812 09:41:04.249414 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:04.249432 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:04.249819 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:04.250090 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:04.250473 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.251428 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.252623 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.253083 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.253217 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:04.254061 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:04.254066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:04.254077 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:41:04.254518 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:04.254573 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:41:04.254594 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:04.254679 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:04.254843 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 09:41:04.255125 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":179,"threads_started":1}
I20260812 09:41:04.256526 14453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:04.257369 14453 log.cc:824] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:04.259346 14453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15: No bootstrap required, opened a new log
I20260812 09:41:04.263124 14453 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8708e7efc6b46d7a2c95dc138918d15" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40889 } }
I20260812 09:41:04.263408 14453 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:04.263521 14453 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8708e7efc6b46d7a2c95dc138918d15, State: Initialized, Role: FOLLOWER
I20260812 09:41:04.264046 14453 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8708e7efc6b46d7a2c95dc138918d15" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40889 } }
I20260812 09:41:04.264212 14453 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:41:04.264278 14453 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:41:04.264387 14453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:04.265247 14453 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8708e7efc6b46d7a2c95dc138918d15" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40889 } }
I20260812 09:41:04.265583 14453 leader_election.cc:302] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8708e7efc6b46d7a2c95dc138918d15; no voters:
I20260812 09:41:04.265872 14453 leader_election.cc:288] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:41:04.266000 14458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:41:04.266244 14458 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [term 1 LEADER]: Becoming Leader. State: Replica: a8708e7efc6b46d7a2c95dc138918d15, State: Running, Role: LEADER
I20260812 09:41:04.266561 14458 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8708e7efc6b46d7a2c95dc138918d15" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40889 } }
I20260812 09:41:04.267143 14453 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:41:04.268322 14460 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8708e7efc6b46d7a2c95dc138918d15. Latest consensus state: current_term: 1 leader_uuid: "a8708e7efc6b46d7a2c95dc138918d15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8708e7efc6b46d7a2c95dc138918d15" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40889 } } }
I20260812 09:41:04.268489 14460 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [sys.catalog]: This master's current role is: LEADER
I20260812 09:41:04.268882 14467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:41:04.268837 14459 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8708e7efc6b46d7a2c95dc138918d15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8708e7efc6b46d7a2c95dc138918d15" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40889 } } }
I20260812 09:41:04.268999 14459 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15 [sys.catalog]: This master's current role is: LEADER
I20260812 09:41:04.272199 14467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:41:04.277074 14467 catalog_manager.cc:1347] Generated new cluster ID: 1b2dd9dc63f943da9c9bc2b35442181e
I20260812 09:41:04.277148 14467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:41:04.284404 14467 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:41:04.285252 14467 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:41:04.294605 14467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8708e7efc6b46d7a2c95dc138918d15: Generated new TSK 0
I20260812 09:41:04.295312 14467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:41:04.298679 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40889
--builtin_ntp_servers=127.3.147.20:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 09:41:04.412196 14478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:04.412446 14478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:04.412485 14478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:04.416663 14478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:04.416715 14478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:04.416795 14478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:41:04.421763 14478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:04.422904 14478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:04.424455 14478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:04.434458 14485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:04.434473 14486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:04.434612 14478 server_base.cc:1034] running on GCE node
W20260812 09:41:04.434453 14488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:04.435237 14478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:04.435892 14478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:04.437098 14478 hybrid_clock.cc:648] HybridClock initialized: now 1786527664437074 us; error 36 us; skew 500 ppm
I20260812 09:41:04.438977 14478 webserver.cc:460] Webserver started at http://127.3.147.1:42307/ using document root <none> and password file <none>
I20260812 09:41:04.439570 14478 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:04.439666 14478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:04.439896 14478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:04.442021 14478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "9e29ec282af24adbb6a296d7cbeff5ca"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "63208e42ee744b53ae3e5751382fda23"
server_key_iv: "4e41d79416fd29a08d285b7b94131bc2"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.442595 14478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "9e29ec282af24adbb6a296d7cbeff5ca"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "63208e42ee744b53ae3e5751382fda23"
server_key_iv: "4e41d79416fd29a08d285b7b94131bc2"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.446256 14478 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:41:04.448801 14495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.449971 14478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:41:04.450099 14478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "9e29ec282af24adbb6a296d7cbeff5ca"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "63208e42ee744b53ae3e5751382fda23"
server_key_iv: "4e41d79416fd29a08d285b7b94131bc2"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.450229 14478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:04.471882 14478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:04.472630 14478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:04.472831 14478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:04.473426 14478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:04.474357 14478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:04.474426 14478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.474490 14478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:04.474520 14478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.482998 14478 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:44553
I20260812 09:41:04.483099 14623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:44553 every 8 connection(s)
I20260812 09:41:04.483891 14478 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:41:04.485899 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 14478
I20260812 09:41:04.486029 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:41:04.486323 3660 external_mini_cluster.cc:1428] Setting key 490aa468c45e617984147d7b1205f009
I20260812 09:41:04.488554 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40889
--builtin_ntp_servers=127.3.147.20:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 09:41:04.493799 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:04.485749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:50613 (local address 127.3.147.62:40889)
0812 09:41:04.485929 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:04.485935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:04.487069 (+ 1134us) server_negotiation.cc:407] Connection header received
0812 09:41:04.488595 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:04.488599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:04.488658 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:04.488740 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:04.489609 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.490383 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.491523 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.491993 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.492119 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:04.493221 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:04.493228 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:04.493234 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:41:04.493252 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:04.493279 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:41:04.493292 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:04.493383 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:04.493492 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 09:41:04.493573 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 09:41:04.494493 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:04.486200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40889 (local address 127.3.147.1:50613)
0812 09:41:04.486915 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:04.486967 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:04.488394 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:04.488907 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:04.488916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:04.488998 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:04.489465 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:04.489477 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.490519 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.490522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:04.491401 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:04.491411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.492179 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.492184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:04.492328 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:04.492966 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:41:04.492968 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:04.493005 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:41:04.493065 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:04.493381 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:04.493388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:04.493392 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 09:41:04.493482 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 09:41:04.493704 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":140,"threads_started":1}
I20260812 09:41:04.495824 14624 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40889
I20260812 09:41:04.496047 14624 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:04.496588 14624 heartbeater.cc:511] Master 127.3.147.62:40889 requested a full tablet report, sending...
I20260812 09:41:04.498004 14413 ts_manager.cc:194] Registered new tserver with Master: 9e29ec282af24adbb6a296d7cbeff5ca (127.3.147.1:44553)
I20260812 09:41:04.499121 14413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.1:50613
W20260812 09:41:04.603243 14631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:04.603526 14631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:04.603600 14631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:04.607652 14631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:04.607723 14631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:04.607815 14631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:41:04.612419 14631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:04.613433 14631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:04.614854 14631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:04.624254 14636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:04.624325 14641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:04.624254 14638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:04.624390 14631 server_base.cc:1034] running on GCE node
I20260812 09:41:04.624951 14631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:04.625550 14631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:04.626717 14631 hybrid_clock.cc:648] HybridClock initialized: now 1786527664626686 us; error 43 us; skew 500 ppm
I20260812 09:41:04.628700 14631 webserver.cc:460] Webserver started at http://127.3.147.2:36501/ using document root <none> and password file <none>
I20260812 09:41:04.629361 14631 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:04.629459 14631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:04.629738 14631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:04.631781 14631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "fcd5958a5eb44c70b18d5d501f17fb80"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "40ac1d91214097152874fa9f70ece27f"
server_key_iv: "8bcba25a71ec72658e5ae62ffef80666"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.632369 14631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "fcd5958a5eb44c70b18d5d501f17fb80"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "40ac1d91214097152874fa9f70ece27f"
server_key_iv: "8bcba25a71ec72658e5ae62ffef80666"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.635921 14631 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:41:04.638413 14648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.639933 14631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:41:04.640051 14631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "fcd5958a5eb44c70b18d5d501f17fb80"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "40ac1d91214097152874fa9f70ece27f"
server_key_iv: "8bcba25a71ec72658e5ae62ffef80666"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.640198 14631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:04.679261 14631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:04.680051 14631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:04.680286 14631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:04.680939 14631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:04.681944 14631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:04.682022 14631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.682116 14631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:04.682165 14631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.691919 14631 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:42205
I20260812 09:41:04.691947 14771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:42205 every 8 connection(s)
I20260812 09:41:04.692925 14631 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:41:04.696015 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 14631
I20260812 09:41:04.696162 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:41:04.696532 3660 external_mini_cluster.cc:1428] Setting key 6a8637bb0b6abd3f025ed0b55ac6c855
I20260812 09:41:04.698757 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40889
--builtin_ntp_servers=127.3.147.20:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 09:41:04.703953 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:04.694893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:36007 (local address 127.3.147.62:40889)
0812 09:41:04.695032 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:04.695036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:04.696303 (+ 1267us) server_negotiation.cc:407] Connection header received
0812 09:41:04.698059 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:04.698062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:04.698125 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:04.698200 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:04.699204 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.700018 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.701747 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.701982 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.702094 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:04.703460 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:04.703467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:04.703470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:41:04.703484 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:04.703506 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:41:04.703515 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:04.703587 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:04.703787 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 09:41:04.703839 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 09:41:04.705389 14774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:04.695318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40889 (local address 127.3.147.2:36007)
0812 09:41:04.696107 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:04.696187 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:04.697824 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:04.698386 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:04.698393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:04.698476 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:04.699080 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:04.699097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.700189 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.700192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:04.701590 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:04.701605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.702166 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.702173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:04.702403 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:04.703107 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:41:04.703112 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:04.703174 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:41:04.703261 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:04.703616 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:04.703627 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:04.703645 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 09:41:04.703791 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 09:41:04.704196 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":118,"threads_started":1}
I20260812 09:41:04.706609 14772 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40889
I20260812 09:41:04.706843 14772 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:04.707347 14772 heartbeater.cc:511] Master 127.3.147.62:40889 requested a full tablet report, sending...
I20260812 09:41:04.708433 14413 ts_manager.cc:194] Registered new tserver with Master: fcd5958a5eb44c70b18d5d501f17fb80 (127.3.147.2:42205)
I20260812 09:41:04.709276 14413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.2:36007
W20260812 09:41:04.814459 14775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:04.814733 14775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:04.814808 14775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:04.818892 14775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:04.818974 14775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:04.819058 14775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:41:04.824002 14775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:04.825026 14775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:04.826390 14775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:04.836112 14783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:04.836162 14782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:04.836374 14775 server_base.cc:1034] running on GCE node
W20260812 09:41:04.836194 14786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:04.836894 14775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:04.837515 14775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:04.838726 14775 hybrid_clock.cc:648] HybridClock initialized: now 1786527664838701 us; error 53 us; skew 500 ppm
I20260812 09:41:04.840849 14775 webserver.cc:460] Webserver started at http://127.3.147.3:40867/ using document root <none> and password file <none>
I20260812 09:41:04.841437 14775 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:04.841526 14775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:04.841771 14775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:04.843891 14775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "2d9b9d6498d2477a85b7f8ca89d56355"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "5858dae5a8ea7adb1e4e9812be2a6b4d"
server_key_iv: "50b6b8a036ef32bbfa8b0736688e26de"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.844444 14775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "2d9b9d6498d2477a85b7f8ca89d56355"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "5858dae5a8ea7adb1e4e9812be2a6b4d"
server_key_iv: "50b6b8a036ef32bbfa8b0736688e26de"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.848079 14775 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:41:04.850595 14793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.851790 14775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:41:04.851925 14775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "2d9b9d6498d2477a85b7f8ca89d56355"
format_stamp: "Formatted at 2026-08-12 09:41:04 on dist-test-slave-cbbz"
server_key: "5858dae5a8ea7adb1e4e9812be2a6b4d"
server_key_iv: "50b6b8a036ef32bbfa8b0736688e26de"
server_key_version: "encryptionkey@0"
I20260812 09:41:04.852033 14775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:04.863564 14775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:04.864327 14775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:04.864521 14775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:04.865075 14775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:04.866056 14775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:04.866127 14775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.866196 14775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:04.866245 14775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:04.875078 14775 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:39475
I20260812 09:41:04.875113 14924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:39475 every 8 connection(s)
I20260812 09:41:04.875989 14775 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:41:04.876228 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 14775
I20260812 09:41:04.876348 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:41:04.876691 3660 external_mini_cluster.cc:1428] Setting key 7272f0cf82c050f13464b23894004167
I20260812 09:41:04.885159 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:04.878027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:59585 (local address 127.3.147.62:40889)
0812 09:41:04.878175 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:04.878179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:04.879391 (+ 1212us) server_negotiation.cc:407] Connection header received
0812 09:41:04.880808 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:04.880810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:04.880857 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:04.880929 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:04.881747 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.882302 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.883388 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.883601 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.883685 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:04.884735 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:04.884738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:04.884740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:41:04.884753 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:04.884772 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:41:04.884779 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:04.884839 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:04.884968 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 09:41:04.885019 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 09:41:04.885995 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:04.878509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40889 (local address 127.3.147.3:59585)
0812 09:41:04.879258 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:04.879298 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:04.880621 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:04.881056 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:04.881062 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:04.881147 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:04.881625 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:04.881637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.882409 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.882411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:04.883282 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:04.883290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:04.883737 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:04.883740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:04.883886 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:04.884495 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:41:04.884497 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:04.884535 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:41:04.884598 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:04.884851 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:04.884858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:04.884861 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 09:41:04.884953 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 09:41:04.885180 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":158,"threads_started":1}
I20260812 09:41:04.887172 14925 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40889
I20260812 09:41:04.887440 14925 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:04.887995 14925 heartbeater.cc:511] Master 127.3.147.62:40889 requested a full tablet report, sending...
I20260812 09:41:04.889146 14413 ts_manager.cc:194] Registered new tserver with Master: 2d9b9d6498d2477a85b7f8ca89d56355 (127.3.147.3:39475)
I20260812 09:41:04.889894 14413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.3:59585
I20260812 09:41:04.890772 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 09:41:04.894029 14456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:04.892745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38864 (local address 127.3.147.62:40889)
0812 09:41:04.892906 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:04.892911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:04.893144 (+ 233us) server_negotiation.cc:407] Connection header received
0812 09:41:04.893305 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:04.893311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:04.893374 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:04.893476 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:04.893931 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38864: BlockingRecv error: recv got EOF from 127.0.0.1:38864 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 09:41:04.894632 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 14478
I20260812 09:41:04.909983 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 14631
I20260812 09:41:04.925531 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 14775
I20260812 09:41:04.932065 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 14377
2026-08-12T09:41:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (915 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T09:41:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:41:04Z Disabled control of system clock
I20260812 09:41:04.965330 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:42735
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:42735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 09:41:05.075501 14938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:05.075773 14938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:05.075811 14938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:05.079789 14938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:41:05.079845 14938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:05.079860 14938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:05.079876 14938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:41:05.079891 14938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:41:05.084623 14938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:42735
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:42735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:05.085638 14938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:05.086937 14938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:05.095404 14946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:05.095422 14938 server_base.cc:1034] running on GCE node
W20260812 09:41:05.095374 14943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:05.095659 14944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:05.096272 14938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:05.097363 14938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:05.098538 14938 hybrid_clock.cc:648] HybridClock initialized: now 1786527665098524 us; error 35 us; skew 500 ppm
I20260812 09:41:05.100317 14938 webserver.cc:460] Webserver started at http://127.3.147.62:46281/ using document root <none> and password file <none>
I20260812 09:41:05.100873 14938 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:05.100961 14938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:05.101192 14938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:05.103188 14938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "b1577d3f5d1941adbf678fed8ad04bb5"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "eeae960981acb82ca2a5a2aba7f9f168"
server_key_iv: "1ca85464340b081c78328cd0f8ec26fa"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.103735 14938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "b1577d3f5d1941adbf678fed8ad04bb5"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "eeae960981acb82ca2a5a2aba7f9f168"
server_key_iv: "1ca85464340b081c78328cd0f8ec26fa"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.107259 14938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 09:41:05.109630 14954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.110946 14938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:41:05.111116 14938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "b1577d3f5d1941adbf678fed8ad04bb5"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "eeae960981acb82ca2a5a2aba7f9f168"
server_key_iv: "1ca85464340b081c78328cd0f8ec26fa"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.111229 14938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:05.115782 14938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:05.116434 14938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:05.116623 14938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:05.123055 14938 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:42735
I20260812 09:41:05.123067 15014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:42735 every 8 connection(s)
I20260812 09:41:05.123919 14938 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:41:05.127133 15015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:05.130688 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 14938
I20260812 09:41:05.130798 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:41:05.131117 3660 external_mini_cluster.cc:1428] Setting key c484bc23ab869206888f88818dd3db42
I20260812 09:41:05.133239 15015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5: Bootstrap starting.
I20260812 09:41:05.135525 15015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:05.136463 15015 log.cc:824] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:05.136945 15018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:05.132685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.3.147.62:42735)
0812 09:41:05.133322 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:05.133357 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:05.133408 (+ 51us) server_negotiation.cc:407] Connection header received
0812 09:41:05.134039 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:05.134055 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:05.134386 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:05.134728 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:05.135514 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:05.135518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:05.135530 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:41:05.135542 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:05.135604 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:41:05.135638 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:05.135745 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:05.135906 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 09:41:05.136182 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":146,"threads_started":1}
I20260812 09:41:05.138543 15015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5: No bootstrap required, opened a new log
I20260812 09:41:05.142040 15015 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1577d3f5d1941adbf678fed8ad04bb5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42735 } }
I20260812 09:41:05.142246 15015 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:05.142287 15015 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1577d3f5d1941adbf678fed8ad04bb5, State: Initialized, Role: FOLLOWER
I20260812 09:41:05.142728 15015 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1577d3f5d1941adbf678fed8ad04bb5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42735 } }
I20260812 09:41:05.142876 15015 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:41:05.142942 15015 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:41:05.143054 15015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:05.144104 15015 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1577d3f5d1941adbf678fed8ad04bb5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42735 } }
I20260812 09:41:05.144465 15015 leader_election.cc:302] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1577d3f5d1941adbf678fed8ad04bb5; no voters:
I20260812 09:41:05.144755 15015 leader_election.cc:288] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:41:05.144989 15020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:41:05.145210 15020 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [term 1 LEADER]: Becoming Leader. State: Replica: b1577d3f5d1941adbf678fed8ad04bb5, State: Running, Role: LEADER
I20260812 09:41:05.145510 15020 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1577d3f5d1941adbf678fed8ad04bb5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42735 } }
I20260812 09:41:05.146044 15015 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:41:05.147169 15023 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1577d3f5d1941adbf678fed8ad04bb5. Latest consensus state: current_term: 1 leader_uuid: "b1577d3f5d1941adbf678fed8ad04bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1577d3f5d1941adbf678fed8ad04bb5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42735 } } }
I20260812 09:41:05.147297 15023 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [sys.catalog]: This master's current role is: LEADER
I20260812 09:41:05.147507 15022 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1577d3f5d1941adbf678fed8ad04bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1577d3f5d1941adbf678fed8ad04bb5" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42735 } } }
I20260812 09:41:05.147670 15022 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5 [sys.catalog]: This master's current role is: LEADER
I20260812 09:41:05.147722 15031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:41:05.150872 15031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:41:05.155869 15031 catalog_manager.cc:1347] Generated new cluster ID: b86fb6191ee44c26bfe1d688afe09aa6
I20260812 09:41:05.155938 15031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:41:05.169441 15031 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:41:05.169761 15031 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:41:05.178416 15031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1577d3f5d1941adbf678fed8ad04bb5: Generated new TSK 0
I20260812 09:41:05.179107 15031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:41:05.185359 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42735
--builtin_ntp_servers=127.3.147.20:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 09:41:05.297175 15041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:05.297422 15041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:05.297463 15041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:05.301546 15041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:05.301602 15041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:05.301657 15041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:41:05.306352 15041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:05.307397 15041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:05.308811 15041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:05.318738 15049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:05.318738 15047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:05.318918 15041 server_base.cc:1034] running on GCE node
W20260812 09:41:05.319023 15046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:05.319538 15041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:05.320142 15041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:05.321344 15041 hybrid_clock.cc:648] HybridClock initialized: now 1786527665321332 us; error 30 us; skew 500 ppm
I20260812 09:41:05.323101 15041 webserver.cc:460] Webserver started at http://127.3.147.1:42389/ using document root <none> and password file <none>
I20260812 09:41:05.323683 15041 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:05.323772 15041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:05.323997 15041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:05.326042 15041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "c4f802112e3b4cc5aa2c09edce5bab70"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "7419705cb6134187c8caf9b272ac321b"
server_key_iv: "fb3fd1f7d6bac3ba54c2dcc09928774e"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.326685 15041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "c4f802112e3b4cc5aa2c09edce5bab70"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "7419705cb6134187c8caf9b272ac321b"
server_key_iv: "fb3fd1f7d6bac3ba54c2dcc09928774e"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.330309 15041 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:41:05.332863 15055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.334064 15041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:41:05.334194 15041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "c4f802112e3b4cc5aa2c09edce5bab70"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "7419705cb6134187c8caf9b272ac321b"
server_key_iv: "fb3fd1f7d6bac3ba54c2dcc09928774e"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.334306 15041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:05.338851 15041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:05.339455 15041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:05.339632 15041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:05.340200 15041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:05.341101 15041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:05.341172 15041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.341236 15041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:05.341281 15041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.350131 15041 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:45697
I20260812 09:41:05.350186 15186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:45697 every 8 connection(s)
I20260812 09:41:05.351017 15041 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:41:05.351814 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 15041
I20260812 09:41:05.351938 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:41:05.352317 3660 external_mini_cluster.cc:1428] Setting key 5e335a769c396bade2e0d39858861831
I20260812 09:41:05.354343 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42735
--builtin_ntp_servers=127.3.147.20:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 09:41:05.357332 15018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:05.352887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:35689 (local address 127.3.147.62:42735)
0812 09:41:05.353009 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:05.353014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:05.354139 (+ 1125us) server_negotiation.cc:407] Connection header received
0812 09:41:05.355518 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:05.355525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:05.355573 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:05.355670 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:05.356824 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:05.356827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:05.356830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:41:05.356843 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:05.356869 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:41:05.356878 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:05.356968 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:05.357152 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 09:41:05.357202 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 09:41:05.358158 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:05.353257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42735 (local address 127.3.147.1:35689)
0812 09:41:05.353976 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:05.354025 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:05.355307 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:05.355821 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:05.355827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:05.355951 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:05.356566 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:41:05.356567 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:05.356613 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:41:05.356678 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:05.356964 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:05.356974 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:05.356983 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 09:41:05.357189 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 09:41:05.357419 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":137,"threads_started":1}
I20260812 09:41:05.359447 15188 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42735
I20260812 09:41:05.359684 15188 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:05.360069 15188 heartbeater.cc:511] Master 127.3.147.62:42735 requested a full tablet report, sending...
I20260812 09:41:05.361740 14973 ts_manager.cc:194] Registered new tserver with Master: c4f802112e3b4cc5aa2c09edce5bab70 (127.3.147.1:45697)
W20260812 09:41:05.468863 15191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:05.469143 15191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:05.469182 15191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:05.473162 15191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:05.473214 15191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:05.473269 15191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:41:05.477943 15191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:05.478966 15191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:05.480347 15191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:41:05.490487 15191 server_base.cc:1034] running on GCE node
W20260812 09:41:05.490646 15196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:05.490540 15201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:05.490484 15197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:05.491226 15191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:05.491927 15191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:05.493101 15191 hybrid_clock.cc:648] HybridClock initialized: now 1786527665493092 us; error 58 us; skew 500 ppm
I20260812 09:41:05.494896 15191 webserver.cc:460] Webserver started at http://127.3.147.2:46565/ using document root <none> and password file <none>
I20260812 09:41:05.495440 15191 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:05.495500 15191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:05.495678 15191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:05.497804 15191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "86633a9c1da74345a4cef524ef43cb4b"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "b6fd6f1d3ea3304235385c96392cebc1"
server_key_iv: "0932ed9be41a1cd0782214579d01f9be"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.498337 15191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "86633a9c1da74345a4cef524ef43cb4b"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "b6fd6f1d3ea3304235385c96392cebc1"
server_key_iv: "0932ed9be41a1cd0782214579d01f9be"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.502229 15191 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:41:05.504761 15209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.505959 15191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:41:05.506057 15191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "86633a9c1da74345a4cef524ef43cb4b"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "b6fd6f1d3ea3304235385c96392cebc1"
server_key_iv: "0932ed9be41a1cd0782214579d01f9be"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.506146 15191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:05.510726 15191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:05.511348 15191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:05.511534 15191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:05.512080 15191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:05.513003 15191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:05.513087 15191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.513175 15191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:05.513224 15191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.522502 15191 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:36437
I20260812 09:41:05.522547 15339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:36437 every 8 connection(s)
I20260812 09:41:05.523701 15191 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:41:05.530133 15018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:05.525652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:41137 (local address 127.3.147.62:42735)
0812 09:41:05.525791 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:05.525796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:05.526941 (+ 1145us) server_negotiation.cc:407] Connection header received
0812 09:41:05.528369 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:05.528373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:05.528442 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:05.528521 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:05.529640 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:05.529643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:05.529646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:41:05.529659 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:05.529682 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:41:05.529692 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:05.529794 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:05.529935 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 09:41:05.529981 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:41:05.530936 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:05.526116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42735 (local address 127.3.147.2:41137)
0812 09:41:05.526793 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:05.526832 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:05.528098 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:05.528691 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:05.528698 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:05.528784 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:05.529382 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:41:05.529384 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:05.529423 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:41:05.529489 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:05.529777 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:05.529786 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:05.529791 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 09:41:05.529967 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 09:41:05.530199 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":119,"threads_started":1}
I20260812 09:41:05.531301 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 15191
I20260812 09:41:05.531443 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:41:05.531762 3660 external_mini_cluster.cc:1428] Setting key 9cd7453714891a681f1276bc1306c1eb
I20260812 09:41:05.532153 15340 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42735
I20260812 09:41:05.532410 15340 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:05.532820 15340 heartbeater.cc:511] Master 127.3.147.62:42735 requested a full tablet report, sending...
I20260812 09:41:05.533816 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42735
--builtin_ntp_servers=127.3.147.20:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 09:41:05.533900 14973 ts_manager.cc:194] Registered new tserver with Master: 86633a9c1da74345a4cef524ef43cb4b (127.3.147.2:36437)
W20260812 09:41:05.667955 15343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:05.668319 15343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:05.668411 15343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:05.672413 15343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:05.672487 15343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:05.672582 15343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:41:05.677277 15343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:05.678431 15343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:05.679855 15343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:05.689544 15351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:05.689548 15348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:05.689549 15349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:05.689872 15343 server_base.cc:1034] running on GCE node
I20260812 09:41:05.690311 15343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:05.690940 15343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:05.692123 15343 hybrid_clock.cc:648] HybridClock initialized: now 1786527665692106 us; error 30 us; skew 500 ppm
I20260812 09:41:05.694093 15343 webserver.cc:460] Webserver started at http://127.3.147.3:43049/ using document root <none> and password file <none>
I20260812 09:41:05.694721 15343 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:05.694835 15343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:05.695104 15343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:05.697331 15343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "8494e0e3c6cf41e1a9bbb48becb3c07c"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "e1afb0deaa2b474b8253e68e9b3890b7"
server_key_iv: "5df81c3781c336a3d516fe3fc842747e"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.697969 15343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "8494e0e3c6cf41e1a9bbb48becb3c07c"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "e1afb0deaa2b474b8253e68e9b3890b7"
server_key_iv: "5df81c3781c336a3d516fe3fc842747e"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.701952 15343 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 09:41:05.704735 15359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.706071 15343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:41:05.706223 15343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "8494e0e3c6cf41e1a9bbb48becb3c07c"
format_stamp: "Formatted at 2026-08-12 09:41:05 on dist-test-slave-cbbz"
server_key: "e1afb0deaa2b474b8253e68e9b3890b7"
server_key_iv: "5df81c3781c336a3d516fe3fc842747e"
server_key_version: "encryptionkey@0"
I20260812 09:41:05.706343 15343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:05.711063 15343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:05.711658 15343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:05.711831 15343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:05.712410 15343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:05.713409 15343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:05.713469 15343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.713531 15343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:05.713557 15343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:05.722735 15343 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:43255
I20260812 09:41:05.722769 15486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:43255 every 8 connection(s)
I20260812 09:41:05.723604 15343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:41:05.729476 15018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:05.725360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:58209 (local address 127.3.147.62:42735)
0812 09:41:05.725495 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:05.725499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:05.726507 (+ 1008us) server_negotiation.cc:407] Connection header received
0812 09:41:05.727778 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:05.727781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:05.727827 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:05.727903 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:05.729093 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:05.729097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:05.729100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:41:05.729112 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:05.729131 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:41:05.729139 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:05.729217 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:05.729336 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 09:41:05.729372 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:41:05.730333 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:05.725725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42735 (local address 127.3.147.3:58209)
0812 09:41:05.726365 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:05.726407 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:05.727579 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:05.728092 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:05.728098 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:05.728222 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:05.728839 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:41:05.728840 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:05.728880 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:41:05.728946 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:05.729218 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:05.729225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:05.729229 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 09:41:05.729358 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 09:41:05.729570 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260812 09:41:05.730921 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 15343
I20260812 09:41:05.731056 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:41:05.731467 3660 external_mini_cluster.cc:1428] Setting key cb859af480016d61a879cca4b112ba9d
I20260812 09:41:05.731643 15487 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42735
I20260812 09:41:05.731868 15487 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:05.732525 15487 heartbeater.cc:511] Master 127.3.147.62:42735 requested a full tablet report, sending...
I20260812 09:41:05.733664 14972 ts_manager.cc:194] Registered new tserver with Master: 8494e0e3c6cf41e1a9bbb48becb3c07c (127.3.147.3:43255)
I20260812 09:41:05.745402 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 09:41:05.748646 15018 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:41:05.747559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.3.147.62:42735)
0812 09:41:05.747674 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:05.747678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:05.747905 (+ 227us) server_negotiation.cc:407] Connection header received
0812 09:41:05.748011 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:05.748013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:05.748045 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:05.748131 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:05.748509 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46262: BlockingRecv error: recv got EOF from 127.0.0.1:46262 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 09:41:05.748757 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 15041
I20260812 09:41:05.764786 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 15191
I20260812 09:41:05.775902 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 15343
I20260812 09:41:05.784198 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 14938
2026-08-12T09:41:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (850 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:41:05 dist-test-slave-cbbz krb5kdc[15497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:41:05 dist-test-slave-cbbz krb5kdc[15497](info): set up 2 sockets
Aug 12 09:41:05 dist-test-slave-cbbz krb5kdc[15497](info): commencing operation
krb5kdc: starting...
W20260812 09:41:06.901276 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:41:06 dist-test-slave-cbbz krb5kdc[15497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:41:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:41:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:41:07.028861 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:39815
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:34389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:39815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:41:07.138314 15510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:07.138622 15510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:07.138696 15510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:07.142743 15510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:41:07.142831 15510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:07.142870 15510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:07.142931 15510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:41:07.142971 15510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:41:07.148229 15510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:34389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:39815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:39815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:07.149230 15510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:07.150254 15510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:07.158812 15519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:07.158809 15518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:07.158890 15510 server_base.cc:1034] running on GCE node
W20260812 09:41:07.159082 15521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:07.159637 15510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:07.160754 15510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:07.161914 15510 hybrid_clock.cc:648] HybridClock initialized: now 1786527667161888 us; error 39 us; skew 500 ppm
Aug 12 09:41:07 dist-test-slave-cbbz krb5kdc[15497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527667, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:41:07.168318 15510 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:41:07.169183 15510 webserver.cc:460] Webserver started at http://127.3.147.62:40177/ using document root <none> and password file <none>
I20260812 09:41:07.169723 15510 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:07.169782 15510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:07.169950 15510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:07.171943 15510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "60bf498c7b624a4ebabef37f8a0a03b2"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "0e06621dac12b71231c7ea318df936bf"
server_key_iv: "bc403fbc077b7cd5ae417344da7a1b2f"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.172523 15510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "60bf498c7b624a4ebabef37f8a0a03b2"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "0e06621dac12b71231c7ea318df936bf"
server_key_iv: "bc403fbc077b7cd5ae417344da7a1b2f"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.176236 15510 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:41:07.178920 15528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.180182 15510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 09:41:07.180315 15510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "60bf498c7b624a4ebabef37f8a0a03b2"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "0e06621dac12b71231c7ea318df936bf"
server_key_iv: "bc403fbc077b7cd5ae417344da7a1b2f"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.180411 15510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:07.192847 15510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:07.195930 15510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:07.196089 15510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:07.202643 15510 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:39815
I20260812 09:41:07.202912 15587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:39815 every 8 connection(s)
I20260812 09:41:07.203554 15510 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:41:07.205389 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 15510
I20260812 09:41:07.205511 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:41:07.205869 3660 external_mini_cluster.cc:1428] Setting key 242c483786389d381bedc01ba7d31c95
I20260812 09:41:07.206633 15588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:07.213510 15588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2: Bootstrap starting.
Aug 12 09:41:07 dist-test-slave-cbbz krb5kdc[15497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:41:07.216008 15588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:07.217031 15588 log.cc:824] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:07.219372 15588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2: No bootstrap required, opened a new log
I20260812 09:41:07.221095 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:07.207205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.3.147.62:39815)
0812 09:41:07.207600 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:07.207635 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:07.207677 (+ 42us) server_negotiation.cc:407] Connection header received
0812 09:41:07.208558 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:07.208580 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:07.208998 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:07.209365 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:07.210066 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.211560 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.212546 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.213229 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.213390 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:07.216036 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:07.216043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:07.216059 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:07.216091 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:07.217938 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:07.218387 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:07.218393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:07.218398 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:07.218551 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:07.218826 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:07.218828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:07.218830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:07.219335 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:07.219542 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:07.219855 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 09:41:07.220175 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260812 09:41:07.223723 15588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bf498c7b624a4ebabef37f8a0a03b2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39815 } }
I20260812 09:41:07.223933 15588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:07.223969 15588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60bf498c7b624a4ebabef37f8a0a03b2, State: Initialized, Role: FOLLOWER
I20260812 09:41:07.224387 15588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bf498c7b624a4ebabef37f8a0a03b2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39815 } }
I20260812 09:41:07.224509 15588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:41:07.224556 15588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:41:07.224628 15588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:07.225445 15588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bf498c7b624a4ebabef37f8a0a03b2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39815 } }
I20260812 09:41:07.225715 15588 leader_election.cc:302] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60bf498c7b624a4ebabef37f8a0a03b2; no voters:
I20260812 09:41:07.225970 15588 leader_election.cc:288] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:41:07.226092 15594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:41:07.226351 15594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [term 1 LEADER]: Becoming Leader. State: Replica: 60bf498c7b624a4ebabef37f8a0a03b2, State: Running, Role: LEADER
I20260812 09:41:07.226722 15594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bf498c7b624a4ebabef37f8a0a03b2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39815 } }
I20260812 09:41:07.227236 15588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:41:07.228391 15595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60bf498c7b624a4ebabef37f8a0a03b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bf498c7b624a4ebabef37f8a0a03b2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39815 } } }
I20260812 09:41:07.228519 15595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:41:07.228905 15596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60bf498c7b624a4ebabef37f8a0a03b2. Latest consensus state: current_term: 1 leader_uuid: "60bf498c7b624a4ebabef37f8a0a03b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bf498c7b624a4ebabef37f8a0a03b2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39815 } } }
I20260812 09:41:07.229076 15596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:41:07.229411 15603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:41:07.232111 15603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:41:07.237303 15603 catalog_manager.cc:1347] Generated new cluster ID: 4b4774d3c5aa4b0e94dad5faf7e844bc
I20260812 09:41:07.237365 15603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:41:07.276912 15603 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:41:07.277892 15603 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:41:07.285126 15603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2: Generated new TSK 0
I20260812 09:41:07.285856 15603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:41:07.316550 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:39815
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:34389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:41:07.431710 15618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:07.432005 15618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:07.432080 15618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:07.436117 15618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:07.436193 15618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:07.436313 15618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:41:07.441644 15618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:34389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:39815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:07.443125 15618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:07.444337 15618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:07.454366 15624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:07.454377 15623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:07.454401 15628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:07.454664 15618 server_base.cc:1034] running on GCE node
I20260812 09:41:07.455175 15618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:07.455744 15618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:07.456903 15618 hybrid_clock.cc:648] HybridClock initialized: now 1786527667456876 us; error 38 us; skew 500 ppm
Aug 12 09:41:07 dist-test-slave-cbbz krb5kdc[15497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527667, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:41:07.459942 15618 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:41:07.460892 15618 webserver.cc:460] Webserver started at http://127.3.147.1:36249/ using document root <none> and password file <none>
I20260812 09:41:07.461467 15618 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:07.461560 15618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:07.461799 15618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:07.463840 15618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "e8236bae71424c2bafbbc0b258f2d377"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "75d4cd49dafed6b21c20434262a6a362"
server_key_iv: "6bb9bf86de6fc41e2b910697e000e762"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.464442 15618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "e8236bae71424c2bafbbc0b258f2d377"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "75d4cd49dafed6b21c20434262a6a362"
server_key_iv: "6bb9bf86de6fc41e2b910697e000e762"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.468034 15618 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 09:41:07.470527 15636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.471721 15618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:41:07.471864 15618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "e8236bae71424c2bafbbc0b258f2d377"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "75d4cd49dafed6b21c20434262a6a362"
server_key_iv: "6bb9bf86de6fc41e2b910697e000e762"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.471971 15618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:07.500375 15618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:07.503687 15618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:07.503896 15618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:07.504524 15618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:07.505353 15618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:07.505447 15618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.505492 15618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:07.505514 15618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.514685 15618 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:42389
I20260812 09:41:07.514729 15772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:42389 every 8 connection(s)
I20260812 09:41:07.515558 15618 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:41:07.523269 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 15618
I20260812 09:41:07.523401 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:41:07.523717 3660 external_mini_cluster.cc:1428] Setting key 5ffee763f0d4fc98360a6968488c8948
Aug 12 09:41:07 dist-test-slave-cbbz krb5kdc[15497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527667, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:41:07.530580 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:07.517352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:49641 (local address 127.3.147.62:39815)
0812 09:41:07.517489 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:07.517493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:07.518349 (+ 856us) server_negotiation.cc:407] Connection header received
0812 09:41:07.519743 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:07.519746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:07.519806 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:07.519917 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:07.521113 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.521715 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.522921 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.523174 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.523286 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:07.526086 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:07.526090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:07.526092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:07.526129 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:07.528023 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:07.528637 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:07.528640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:07.528642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:07.528761 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:07.529159 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:07.529165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:07.529169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:07.529442 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:07.529571 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:07.530163 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 09:41:07.530375 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:41:07.531471 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:07.517714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:39815 (local address 127.3.147.1:49641)
0812 09:41:07.518194 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:07.518236 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:07.519513 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:07.520093 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:07.520100 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:07.520485 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:07.520950 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:07.520963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.521862 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.521866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:07.522788 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:07.522798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.523342 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.523346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:07.523563 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:07.524251 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:07.524267 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:07.525919 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:07.528195 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:07.528201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:07.528212 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:07.528529 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:07.528888 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:07.528891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:07.528893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:07.529034 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:07.529591 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:07.529595 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:07.529821 (+ 226us) client_negotiation.cc:769] Sending connection context
0812 09:41:07.530038 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 09:41:07.530370 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":118,"threads_started":1}
I20260812 09:41:07.533340 15773 heartbeater.cc:347] Connected to a master server at 127.3.147.62:39815
I20260812 09:41:07.533715 15773 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:07.534346 15773 heartbeater.cc:511] Master 127.3.147.62:39815 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:41:07.536041 15548 ts_manager.cc:194] Registered new tserver with Master: e8236bae71424c2bafbbc0b258f2d377 (127.3.147.1:42389)
I20260812 09:41:07.537194 15548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:49641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:41:07.555346 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:39815
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:34389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:41:07.667086 15779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:07.667330 15779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:07.667372 15779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:07.671360 15779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:07.671406 15779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:07.671489 15779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:41:07.676641 15779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:34389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:39815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:07.677614 15779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:07.678632 15779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:07.688591 15792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:07.688591 15787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:07.688599 15789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:07.688805 15779 server_base.cc:1034] running on GCE node
I20260812 09:41:07.689335 15779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:07.689982 15779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:07.691190 15779 hybrid_clock.cc:648] HybridClock initialized: now 1786527667691171 us; error 36 us; skew 500 ppm
Aug 12 09:41:07 dist-test-slave-cbbz krb5kdc[15497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527667, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:41:07.694094 15779 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:41:07.694983 15779 webserver.cc:460] Webserver started at http://127.3.147.2:42627/ using document root <none> and password file <none>
I20260812 09:41:07.695565 15779 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:07.695667 15779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:07.695919 15779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:07.698125 15779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "e08e60e04b4841c29e7181b306d8bde8"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "abc979c6729021779df05ded93ff16a7"
server_key_iv: "187fd31128f2860ed4d26055fbdabbcd"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.698751 15779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "e08e60e04b4841c29e7181b306d8bde8"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "abc979c6729021779df05ded93ff16a7"
server_key_iv: "187fd31128f2860ed4d26055fbdabbcd"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.702710 15779 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 09:41:07.705331 15800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.706779 15779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 09:41:07.706941 15779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "e08e60e04b4841c29e7181b306d8bde8"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "abc979c6729021779df05ded93ff16a7"
server_key_iv: "187fd31128f2860ed4d26055fbdabbcd"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.707042 15779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:07.733562 15779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:07.737241 15779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:07.737411 15779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:07.737995 15779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:07.738854 15779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:07.738907 15779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.738952 15779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:07.738969 15779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.748687 15779 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:41369
I20260812 09:41:07.748711 15929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:41369 every 8 connection(s)
I20260812 09:41:07.749662 15779 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:41:07.752097 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 15779
I20260812 09:41:07.752247 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:41:07.752622 3660 external_mini_cluster.cc:1428] Setting key 81e353ec58ba0b5db7da77c7b9d53c8d
Aug 12 09:41:07 dist-test-slave-cbbz krb5kdc[15497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527667, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:41:07.766460 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:07.751604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:34931 (local address 127.3.147.62:39815)
0812 09:41:07.751808 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:07.751814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:07.752944 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 09:41:07.754693 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:07.754696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:07.754741 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:07.754815 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:07.756491 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.757475 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.758698 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.759204 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.759335 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:07.762268 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:07.762274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:07.762277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:07.762311 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:07.764119 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:07.764733 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:07.764740 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:07.764746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:07.764853 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:07.765197 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:07.765200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:07.765201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:07.765374 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:07.765452 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:07.766209 (+ 757us) server_negotiation.cc:299] Negotiation successful
0812 09:41:07.766333 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:41:07.767468 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:07.752039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:39815 (local address 127.3.147.2:34931)
0812 09:41:07.752751 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:07.752811 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:07.754459 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:07.754986 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:07.754997 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:07.755577 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:07.756322 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:07.756348 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.757645 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.757648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:07.758550 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:07.758560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.759421 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.759427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:07.759607 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:07.760308 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:07.760324 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:07.762084 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:07.764270 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:07.764275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:07.764286 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:07.764608 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:07.764941 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:07.764944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:07.764945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:07.765090 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:07.765545 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:07.765550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:07.765925 (+ 375us) client_negotiation.cc:769] Sending connection context
0812 09:41:07.766095 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 09:41:07.766393 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":175,"threads_started":1}
I20260812 09:41:07.768708 15930 heartbeater.cc:347] Connected to a master server at 127.3.147.62:39815
I20260812 09:41:07.768959 15930 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:07.769572 15930 heartbeater.cc:511] Master 127.3.147.62:39815 requested a full tablet report, sending...
I20260812 09:41:07.770744 15548 ts_manager.cc:194] Registered new tserver with Master: e08e60e04b4841c29e7181b306d8bde8 (127.3.147.2:41369)
I20260812 09:41:07.771634 15548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:34931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:41:07.783499 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:39815
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:34389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:41:07.895035 15936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:07.895294 15936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:07.895336 15936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:07.899313 15936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:07.899362 15936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:07.899446 15936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:41:07.904554 15936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:34389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:39815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:07.905493 15936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:07.906476 15936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:07.915921 15945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:07.915917 15943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:07.915915 15942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:07.916216 15936 server_base.cc:1034] running on GCE node
I20260812 09:41:07.916673 15936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:07.917294 15936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:07.918476 15936 hybrid_clock.cc:648] HybridClock initialized: now 1786527667918461 us; error 38 us; skew 500 ppm
Aug 12 09:41:07 dist-test-slave-cbbz krb5kdc[15497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527667, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:41:07.921473 15936 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:41:07.922521 15936 webserver.cc:460] Webserver started at http://127.3.147.3:35955/ using document root <none> and password file <none>
I20260812 09:41:07.923184 15936 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:07.923287 15936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:07.923537 15936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:41:07.925693 15936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "1f27da038cee4a78a209abdb629466df"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "57b17c272cafb5b4cce496c41af1919c"
server_key_iv: "f404e26b39e308ed6998dca529d9e6de"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.926301 15936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "1f27da038cee4a78a209abdb629466df"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "57b17c272cafb5b4cce496c41af1919c"
server_key_iv: "f404e26b39e308ed6998dca529d9e6de"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.930493 15936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:41:07.932996 15953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.934252 15936 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:41:07.934403 15936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "1f27da038cee4a78a209abdb629466df"
format_stamp: "Formatted at 2026-08-12 09:41:07 on dist-test-slave-cbbz"
server_key: "57b17c272cafb5b4cce496c41af1919c"
server_key_iv: "f404e26b39e308ed6998dca529d9e6de"
server_key_version: "encryptionkey@0"
I20260812 09:41:07.934525 15936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:07.953449 15936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:07.956868 15936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:07.957072 15936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:07.957687 15936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:07.958511 15936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:07.958592 15936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.958712 15936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:07.958765 15936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:07.967911 15936 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:44103
I20260812 09:41:07.967964 16083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:44103 every 8 connection(s)
I20260812 09:41:07.968833 15936 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:41:07.969894 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 15936
I20260812 09:41:07.970013 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:41:07.970304 3660 external_mini_cluster.cc:1428] Setting key 7d9b560d06859f9ee6cebcee30dbbbb6
Aug 12 09:41:07 dist-test-slave-cbbz krb5kdc[15497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527667, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:41:07.982230 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:07.970827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:42547 (local address 127.3.147.62:39815)
0812 09:41:07.970976 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:07.970981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:07.971893 (+ 912us) server_negotiation.cc:407] Connection header received
0812 09:41:07.973506 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:07.973508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:07.973553 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:07.973621 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:07.974697 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.975251 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.976490 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.976702 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.976783 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:07.979069 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:07.979072 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:07.979074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:07.979099 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:07.980446 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:07.980895 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:07.980898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:07.980899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:07.980972 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:07.981289 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:07.981291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:07.981292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:07.981459 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:07.981534 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:07.982007 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 09:41:07.982112 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:41:07.983027 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:07.971278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:39815 (local address 127.3.147.3:42547)
0812 09:41:07.971742 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:07.971793 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:07.973304 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:07.973744 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:07.973750 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:07.974086 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:07.974523 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:07.974535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.975385 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.975389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:07.976382 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:07.976393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.976831 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:07.976834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:07.976967 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:07.977451 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:07.977466 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:07.978920 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:07.980550 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:07.980554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:07.980565 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:07.980808 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:07.981054 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:07.981056 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:07.981057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:07.981207 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:07.981562 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:07.981567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:07.981791 (+ 224us) client_negotiation.cc:769] Sending connection context
0812 09:41:07.981958 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 09:41:07.982216 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":129,"threads_started":1}
I20260812 09:41:07.984196 16084 heartbeater.cc:347] Connected to a master server at 127.3.147.62:39815
I20260812 09:41:07.984439 16084 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:07.984953 16084 heartbeater.cc:511] Master 127.3.147.62:39815 requested a full tablet report, sending...
I20260812 09:41:07.985970 15548 ts_manager.cc:194] Registered new tserver with Master: 1f27da038cee4a78a209abdb629466df (127.3.147.3:44103)
I20260812 09:41:07.986546 15548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:42547
I20260812 09:41:07.996219 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:41:08.005091 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:07.998075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.3.147.62:39815)
0812 09:41:07.998232 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:07.998237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:07.998419 (+ 182us) server_negotiation.cc:407] Connection header received
0812 09:41:07.998483 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:07.998485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:07.998523 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:07.998589 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:07.999253 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:07.999780 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.000728 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.000923 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.000999 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.002314 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:08.002316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:08.002318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:08.002339 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:08.003514 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:08.003888 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:08.003891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:08.003892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:08.003962 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:08.004241 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:08.004244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:08.004245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:08.004450 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:08.004564 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.004788 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.004907 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 09:41:08.007705 15548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:41:08.010793 15548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:41:08.025334 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.018996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:41369 (local address 127.0.0.1:35718)
0812 09:41:08.020289 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.020303 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:08.020393 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:08.020834 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:08.020843 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:08.020870 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:08.021091 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.021099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.022328 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.022334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.023595 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.023612 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.024746 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.024752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.024863 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.024901 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 09:41:08.025022 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 09:41:08.025174 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1231,"mutex_wait_us":50,"spinlock_wait_cycles":4736,"thread_start_us":91,"threads_started":1}
I20260812 09:41:08.025373 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.019872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35718 (local address 127.3.147.2:41369)
0812 09:41:08.020293 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:08.020311 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:08.020371 (+ 60us) server_negotiation.cc:407] Connection header received
0812 09:41:08.020503 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:08.020508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:08.020707 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:08.020825 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:08.021251 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.022197 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.023729 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.024612 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.024749 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.024899 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.025081 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.025207 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":125,"threads_started":1}
I20260812 09:41:08.025501 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.019042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.3.147.3:44103)
0812 09:41:08.019601 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:08.019616 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:08.020045 (+ 429us) server_negotiation.cc:407] Connection header received
0812 09:41:08.020325 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:08.020334 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:08.020557 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:08.020691 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:08.021220 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.022406 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.023917 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.024680 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.024769 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.024899 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.025258 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.025353 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":210,"threads_started":1}
I20260812 09:41:08.025846 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.019625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:42389 (local address 127.0.0.1:54338)
0812 09:41:08.020589 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.020603 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:08.020690 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:08.021486 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:08.021490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:08.021510 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:08.021624 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.021631 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.023027 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.023030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.024212 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.024220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.025289 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.025292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.025428 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.025545 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 09:41:08.025623 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 09:41:08.025690 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":94,"threads_started":1}
I20260812 09:41:08.025895 16095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.018892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:44103 (local address 127.0.0.1:58474)
0812 09:41:08.019868 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.019919 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:08.020076 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:08.020890 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:08.020892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:08.020915 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:08.021091 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.021099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.022557 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.022588 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.023781 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.023792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.024971 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.024974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.025096 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.025119 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:41:08.025663 (+ 544us) client_negotiation.cc:240] Negotiation successful
0812 09:41:08.025758 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":38,"spinlock_wait_cycles":384,"thread_start_us":71,"threads_started":1}
I20260812 09:41:08.027063 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.019853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.3.147.1:42389)
0812 09:41:08.021040 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:08.021052 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:08.021068 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:41:08.021138 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:08.021141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:08.021335 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:08.021484 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:08.021753 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.022877 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.024321 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.024986 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.025100 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.025229 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.026804 (+ 1575us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.026949 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1098,"thread_start_us":128,"threads_started":1}
I20260812 09:41:08.029136 15858 tablet_service.cc:1467] Processing CreateTablet for tablet 930c200102bb44c083bc9b2eb7083741 (DEFAULT_TABLE table=test-table [id=ea64ef5111d748ac896020454d56e027]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:41:08.029134 16009 tablet_service.cc:1467] Processing CreateTablet for tablet 930c200102bb44c083bc9b2eb7083741 (DEFAULT_TABLE table=test-table [id=ea64ef5111d748ac896020454d56e027]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:41:08.029719 15694 tablet_service.cc:1467] Processing CreateTablet for tablet 930c200102bb44c083bc9b2eb7083741 (DEFAULT_TABLE table=test-table [id=ea64ef5111d748ac896020454d56e027]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:41:08.030357 15858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930c200102bb44c083bc9b2eb7083741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:08.030359 16009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930c200102bb44c083bc9b2eb7083741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:08.030551 15694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 930c200102bb44c083bc9b2eb7083741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:08.037518 16102 tablet_bootstrap.cc:492] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377: Bootstrap starting.
I20260812 09:41:08.038405 16104 tablet_bootstrap.cc:492] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: Bootstrap starting.
I20260812 09:41:08.039896 16102 tablet_bootstrap.cc:654] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:08.040375 16104 tablet_bootstrap.cc:654] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:08.040392 16103 tablet_bootstrap.cc:492] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8: Bootstrap starting.
I20260812 09:41:08.040758 16102 log.cc:824] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:08.041772 16104 log.cc:824] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:08.042809 16102 tablet_bootstrap.cc:492] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377: No bootstrap required, opened a new log
I20260812 09:41:08.043002 16102 ts_tablet_manager.cc:1397] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260812 09:41:08.043144 16103 tablet_bootstrap.cc:654] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:08.044000 16103 log.cc:824] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:08.044230 16104 tablet_bootstrap.cc:492] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: No bootstrap required, opened a new log
I20260812 09:41:08.044428 16104 ts_tablet_manager.cc:1397] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:41:08.046037 16103 tablet_bootstrap.cc:492] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8: No bootstrap required, opened a new log
I20260812 09:41:08.046280 16103 ts_tablet_manager.cc:1397] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:41:08.046608 16102 raft_consensus.cc:348] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.046849 16102 raft_consensus.cc:374] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:08.046901 16102 raft_consensus.cc:729] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8236bae71424c2bafbbc0b258f2d377, State: Initialized, Role: FOLLOWER
I20260812 09:41:08.047370 16102 consensus_queue.cc:260] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.048020 15773 heartbeater.cc:503] Master 127.3.147.62:39815 was elected leader, sending a full tablet report...
I20260812 09:41:08.048401 16102 ts_tablet_manager.cc:1428] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260812 09:41:08.048381 16104 raft_consensus.cc:348] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.048630 16104 raft_consensus.cc:374] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:08.048688 16104 raft_consensus.cc:729] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f27da038cee4a78a209abdb629466df, State: Initialized, Role: FOLLOWER
I20260812 09:41:08.049134 16104 consensus_queue.cc:260] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.050408 16084 heartbeater.cc:503] Master 127.3.147.62:39815 was elected leader, sending a full tablet report...
I20260812 09:41:08.050530 16103 raft_consensus.cc:348] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.050777 16103 raft_consensus.cc:374] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:08.050841 16103 raft_consensus.cc:729] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08e60e04b4841c29e7181b306d8bde8, State: Initialized, Role: FOLLOWER
I20260812 09:41:08.051246 16103 consensus_queue.cc:260] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.051692 16104 ts_tablet_manager.cc:1428] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 09:41:08.052222 16103 ts_tablet_manager.cc:1428] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:41:08.052585 15930 heartbeater.cc:503] Master 127.3.147.62:39815 was elected leader, sending a full tablet report...
I20260812 09:41:08.090821 16111 raft_consensus.cc:482] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:41:08.091069 16111 raft_consensus.cc:504] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.091908 16110 raft_consensus.cc:482] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:41:08.092124 16110 raft_consensus.cc:504] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.092383 16111 leader_election.cc:288] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f27da038cee4a78a209abdb629466df (127.3.147.3:44103), e8236bae71424c2bafbbc0b258f2d377 (127.3.147.1:42389)
I20260812 09:41:08.093389 16110 leader_election.cc:288] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e08e60e04b4841c29e7181b306d8bde8 (127.3.147.2:41369), e8236bae71424c2bafbbc0b258f2d377 (127.3.147.1:42389)
I20260812 09:41:08.097900 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.093517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:41369 (local address 127.3.147.3:45433)
0812 09:41:08.093709 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.093723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:08.093796 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:08.094078 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:08.094081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:08.094098 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:08.094211 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.094217 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.095204 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.095207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.096463 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.096474 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.097429 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.097433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.097527 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.097565 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 09:41:08.097664 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 09:41:08.097750 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260812 09:41:08.097999 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.092655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:44103 (local address 127.3.147.2:45113)
0812 09:41:08.092808 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.092825 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:08.092944 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:08.093370 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:08.093373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:08.093396 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:08.093525 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.093534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.095004 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.095007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.096276 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.096286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.097557 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.097560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.097698 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.097720 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 09:41:08.097784 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 09:41:08.097852 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":25}
I20260812 09:41:08.098394 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.093752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:45433 (local address 127.3.147.2:41369)
0812 09:41:08.093877 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:08.093882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:08.093895 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:41:08.093935 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:08.093938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:08.093980 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:08.094059 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:08.094333 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.095067 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.096625 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.097229 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.097316 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.097374 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.098200 (+ 826us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.098269 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":39}
I20260812 09:41:08.098435 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.092733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:45113 (local address 127.3.147.3:44103)
0812 09:41:08.092856 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:08.092862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:08.092878 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:41:08.093099 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:08.093102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:08.093155 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:08.093233 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:08.094156 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.094884 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.096806 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.097343 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.097444 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.097502 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.098244 (+ 742us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.098314 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 09:41:08.098901 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.093753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:44101 (local address 127.3.147.1:42389)
0812 09:41:08.094101 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:08.094111 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:08.094130 (+ 19us) server_negotiation.cc:407] Connection header received
0812 09:41:08.094185 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:08.094188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:08.094244 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:08.094340 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:08.094715 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.095523 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.096975 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.097577 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.097749 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.097805 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.098694 (+ 889us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.098766 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":268,"thread_start_us":95,"threads_started":1}
I20260812 09:41:08.098912 15881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930c200102bb44c083bc9b2eb7083741" candidate_uuid: "1f27da038cee4a78a209abdb629466df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e08e60e04b4841c29e7181b306d8bde8" is_pre_election: true
I20260812 09:41:08.098913 16031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930c200102bb44c083bc9b2eb7083741" candidate_uuid: "e08e60e04b4841c29e7181b306d8bde8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f27da038cee4a78a209abdb629466df" is_pre_election: true
I20260812 09:41:08.099108 15881 raft_consensus.cc:2454] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f27da038cee4a78a209abdb629466df in term 0.
I20260812 09:41:08.099133 16031 raft_consensus.cc:2454] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e08e60e04b4841c29e7181b306d8bde8 in term 0.
I20260812 09:41:08.099287 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.093576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:42389 (local address 127.3.147.3:44101)
0812 09:41:08.093978 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.093992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:08.094087 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:08.094452 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:08.094455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:08.094471 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:08.094589 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.094595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.095653 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.095656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.096848 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.096858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.097848 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.097851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.098275 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.098299 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 09:41:08.098368 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 09:41:08.099049 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":109,"threads_started":1}
I20260812 09:41:08.099560 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.093168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:50491 (local address 127.3.147.1:42389)
0812 09:41:08.093367 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:08.093372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:08.094186 (+ 814us) server_negotiation.cc:407] Connection header received
0812 09:41:08.094476 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:08.094479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:08.094523 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:08.094565 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:08.094900 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.095655 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.097695 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.098302 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.098374 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.098424 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.099352 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.099453 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260812 09:41:08.099604 15804 leader_election.cc:302] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f27da038cee4a78a209abdb629466df, e08e60e04b4841c29e7181b306d8bde8; no voters:
I20260812 09:41:08.099642 15955 leader_election.cc:302] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f27da038cee4a78a209abdb629466df, e08e60e04b4841c29e7181b306d8bde8; no voters:
I20260812 09:41:08.099864 15719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930c200102bb44c083bc9b2eb7083741" candidate_uuid: "1f27da038cee4a78a209abdb629466df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8236bae71424c2bafbbc0b258f2d377" is_pre_election: true
I20260812 09:41:08.099925 16110 raft_consensus.cc:2781] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:41:08.099963 16111 raft_consensus.cc:2781] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:41:08.100036 16110 raft_consensus.cc:482] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:41:08.100044 16111 raft_consensus.cc:482] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:41:08.100104 16111 raft_consensus.cc:3037] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:08.100106 16110 raft_consensus.cc:3037] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:08.100221 15719 raft_consensus.cc:2454] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f27da038cee4a78a209abdb629466df in term 0.
I20260812 09:41:08.100541 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.092973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:42389 (local address 127.3.147.2:50491)
0812 09:41:08.093966 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.093979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:08.094354 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:08.094677 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:08.094680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:08.094695 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:08.094791 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.094795 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.095794 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.095797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.097514 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:08.097527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.098549 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.098554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:08.098653 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.098675 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 09:41:08.100232 (+ 1557us) client_negotiation.cc:240] Negotiation successful
0812 09:41:08.100369 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":531,"thread_start_us":154,"threads_started":1}
I20260812 09:41:08.101081 15719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930c200102bb44c083bc9b2eb7083741" candidate_uuid: "e08e60e04b4841c29e7181b306d8bde8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8236bae71424c2bafbbc0b258f2d377" is_pre_election: true
I20260812 09:41:08.101199 15719 raft_consensus.cc:2454] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e08e60e04b4841c29e7181b306d8bde8 in term 0.
I20260812 09:41:08.101572 16110 raft_consensus.cc:504] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.101598 16111 raft_consensus.cc:504] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.101958 16110 leader_election.cc:288] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [CANDIDATE]: Term 1 election: Requested vote from peers e08e60e04b4841c29e7181b306d8bde8 (127.3.147.2:41369), e8236bae71424c2bafbbc0b258f2d377 (127.3.147.1:42389)
I20260812 09:41:08.101995 16111 leader_election.cc:288] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [CANDIDATE]: Term 1 election: Requested vote from peers 1f27da038cee4a78a209abdb629466df (127.3.147.3:44103), e8236bae71424c2bafbbc0b258f2d377 (127.3.147.1:42389)
I20260812 09:41:08.102324 15719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930c200102bb44c083bc9b2eb7083741" candidate_uuid: "e08e60e04b4841c29e7181b306d8bde8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8236bae71424c2bafbbc0b258f2d377"
I20260812 09:41:08.102332 16031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930c200102bb44c083bc9b2eb7083741" candidate_uuid: "e08e60e04b4841c29e7181b306d8bde8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f27da038cee4a78a209abdb629466df"
I20260812 09:41:08.102445 15719 raft_consensus.cc:3037] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:08.102519 16031 raft_consensus.cc:2379] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e08e60e04b4841c29e7181b306d8bde8 in current term 1: Already voted for candidate 1f27da038cee4a78a209abdb629466df in this term.
I20260812 09:41:08.102566 15881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930c200102bb44c083bc9b2eb7083741" candidate_uuid: "1f27da038cee4a78a209abdb629466df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e08e60e04b4841c29e7181b306d8bde8"
I20260812 09:41:08.102735 15881 raft_consensus.cc:2379] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f27da038cee4a78a209abdb629466df in current term 1: Already voted for candidate e08e60e04b4841c29e7181b306d8bde8 in this term.
I20260812 09:41:08.102458 15718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "930c200102bb44c083bc9b2eb7083741" candidate_uuid: "1f27da038cee4a78a209abdb629466df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8236bae71424c2bafbbc0b258f2d377"
I20260812 09:41:08.103425 15956 leader_election.cc:302] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f27da038cee4a78a209abdb629466df; no voters: e08e60e04b4841c29e7181b306d8bde8, e8236bae71424c2bafbbc0b258f2d377
I20260812 09:41:08.103616 16110 raft_consensus.cc:2726] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 09:41:08.103813 15719 raft_consensus.cc:2454] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08e60e04b4841c29e7181b306d8bde8 in term 1.
I20260812 09:41:08.104065 15804 leader_election.cc:302] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e08e60e04b4841c29e7181b306d8bde8, e8236bae71424c2bafbbc0b258f2d377; no voters: 1f27da038cee4a78a209abdb629466df
I20260812 09:41:08.104290 16111 raft_consensus.cc:2781] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:41:08.104511 16111 raft_consensus.cc:686] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 1 LEADER]: Becoming Leader. State: Replica: e08e60e04b4841c29e7181b306d8bde8, State: Running, Role: LEADER
I20260812 09:41:08.104821 16111 consensus_queue.cc:237] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } }
I20260812 09:41:08.107748 15548 catalog_manager.cc:5495] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e08e60e04b4841c29e7181b306d8bde8 (127.3.147.2). New cstate: current_term: 1 leader_uuid: "e08e60e04b4841c29e7181b306d8bde8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e08e60e04b4841c29e7181b306d8bde8" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260812 09:41:08.124050 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.120412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35732 (local address 127.3.147.2:41369)
0812 09:41:08.120559 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:08.120563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:08.120576 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:41:08.120740 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:08.120748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:08.120797 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:08.120901 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:41:08.121161 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.121640 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.122501 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:08.122848 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:08.122943 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:08.123271 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:41:08.123668 (+ 397us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:41:08.123812 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:08.123874 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 09:41:08.123933 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 09:41:08.129966 16031 raft_consensus.cc:1261] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 1 FOLLOWER]: Refusing update from remote peer e08e60e04b4841c29e7181b306d8bde8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:41:08.129964 15719 raft_consensus.cc:1261] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377 [term 1 FOLLOWER]: Refusing update from remote peer e08e60e04b4841c29e7181b306d8bde8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:41:08.130556 16115 consensus_queue.cc:1035] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f27da038cee4a78a209abdb629466df" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:41:08.130759 16111 consensus_queue.cc:1035] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8236bae71424c2bafbbc0b258f2d377" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:41:08.139029 16120 mvcc.cc:205] Tried to move back new op lower bound from 7317617328653463552 to 7317617328560046080. Current Snapshot: MvccSnapshot[applied={T|T < 7317617328653463552}]
I20260812 09:41:08.148275 15548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:41:08.148418 15548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:41:08.150674 15548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60bf498c7b624a4ebabef37f8a0a03b2: Sending DeleteTablet for 3 replicas of tablet 930c200102bb44c083bc9b2eb7083741
I20260812 09:41:08.151484 16009 tablet_service.cc:1514] Processing DeleteTablet for tablet 930c200102bb44c083bc9b2eb7083741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:41:08 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:58474
I20260812 09:41:08.151489 15694 tablet_service.cc:1514] Processing DeleteTablet for tablet 930c200102bb44c083bc9b2eb7083741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:41:08 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:54338
I20260812 09:41:08.151990 15858 tablet_service.cc:1514] Processing DeleteTablet for tablet 930c200102bb44c083bc9b2eb7083741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:41:08 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35718
I20260812 09:41:08.152199 16129 tablet_replica.cc:330] T 930c200102bb44c083bc9b2eb7083741 P e8236bae71424c2bafbbc0b258f2d377: stopping tablet replica
I20260812 09:41:08.152396 16128 tablet_replica.cc:330] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: stopping tablet replica
I20260812 09:41:08.152571 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 15618
I20260812 09:41:08.152722 16128 raft_consensus.cc:2229] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:41:08.153041 16128 raft_consensus.cc:2258] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:41:08.158931 16128 ts_tablet_manager.cc:1905] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:41:08.160212 16130 tablet_replica.cc:330] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8: stopping tablet replica
I20260812 09:41:08.160611 16130 raft_consensus.cc:2229] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:41:08.161089 16130 raft_consensus.cc:2258] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:41:08.162277 16128 ts_tablet_manager.cc:1918] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:41:08.162381 16128 log.cc:1196] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/930c200102bb44c083bc9b2eb7083741
I20260812 09:41:08.162736 16128 ts_tablet_manager.cc:1939] T 930c200102bb44c083bc9b2eb7083741 P 1f27da038cee4a78a209abdb629466df: Deleting consensus metadata
I20260812 09:41:08.162747 16130 ts_tablet_manager.cc:1905] T 930c200102bb44c083bc9b2eb7083741 P e08e60e04b4841c29e7181b306d8bde8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:41:08.164054 15531 catalog_manager.cc:4841] TS 1f27da038cee4a78a209abdb629466df (127.3.147.3:44103): tablet 930c200102bb44c083bc9b2eb7083741 (table test-table [id=ea64ef5111d748ac896020454d56e027]) successfully deleted
W20260812 09:41:08.164366 15531 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:42389 (error 108)
I20260812 09:41:08.164567 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 15779
I20260812 09:41:08.169026 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.168756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:42389 (local address 127.0.0.1:54352)
0812 09:41:08.168891 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.168954 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 09:41:08.169286 15531 catalog_manager.cc:4584] TS e8236bae71424c2bafbbc0b258f2d377 (127.3.147.1:42389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 930c200102bb44c083bc9b2eb7083741: Network error: Client connection negotiation failed: client connection to 127.3.147.1:42389: connect: Connection refused (error 111)
I20260812 09:41:08.173357 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 15936
I20260812 09:41:08.173774 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:08.173533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:41369 (local address 127.0.0.1:35742)
0812 09:41:08.173676 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:08.173714 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.2:41369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 09:41:08.180261 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 15510
2026-08-12T09:41:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2397 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:41:08 dist-test-slave-cbbz krb5kdc[16134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:41:08 dist-test-slave-cbbz krb5kdc[16134](info): set up 2 sockets
Aug 12 09:41:08 dist-test-slave-cbbz krb5kdc[16134](info): commencing operation
krb5kdc: starting...
W20260812 09:41:09.299885 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:41:09 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:41:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:41:09Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.147.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.147.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/rangeradmin_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/rangeradmin_127.3.147.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.147.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.147.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/rangerlookup_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/rangerlookup_127.3.147.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.147.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.147.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/HTTP_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/HTTP_127.3.147.20@KRBTEST.COM.keytab.
I20260812 09:41:09.466305 3660 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknDSHUD/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/postgres -l logfile start
2026-08-12 09:41:12.047 UTC [16168] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 09:41:12.047 UTC [16168] LOG: listening on IPv4 address "127.3.147.20", port 36571
2026-08-12 09:41:12.051 UTC [16168] LOG: listening on Unix socket "/tmp/.s.PGSQL.36571"
2026-08-12 09:41:12.075 UTC [16171] LOG: database system was shut down at 2026-08-12 09:41:10 UTC
2026-08-12 09:41:12.079 UTC [16168] LOG: database system is ready to accept connections
I20260812 09:41:13.100818 3660 mini_postgres.cc:96] Postgres bound to 36571
2026-08-12 09:41:13.108 UTC [16182] FATAL: database "slave" does not exist
127.3.147.20:36571 - accepting connections
I20260812 09:41:13.109716 3660 mini_ranger.cc:161] Starting Ranger...
I20260812 09:41:13.125926 3660 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 09:41:13.609591 3660 mini_ranger.cc:87] Created ranger Postgres database
I20260812 09:41:13.609694 3660 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-admin
2026-08-12 09:41:13,993 [I] DB FLAVOR :POSTGRES
2026-08-12 09:41:13,993 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 09:41:13,994 [I] Checking connection..
2026-08-12 09:41:13,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 09:41:14,283 [I] Checking connection passed.
2026-08-12 09:41:14,284 [I] --------- Verifying version history table ---------
2026-08-12 09:41:14,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 09:41:14,539 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 09:41:14,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 09:41:14,808 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 09:41:14,808 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 09:41:14,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 09:41:15.058 UTC [16252] WARNING: there is no transaction in progress
2026-08-12 09:41:15,066 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 09:41:15,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 09:41:15,316 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 09:41:15,317 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 09:41:15,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 09:41:15,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 09:41:15,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cbbz.c.gcp-upstream.internal','N') ;"
2026-08-12 09:41:16,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 09:41:16,352 [I] Table x_portal_user does not exist in database ranger
2026-08-12 09:41:16,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 09:41:16,625 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 09:41:16,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 09:41:16,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 09:41:17,160 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 09:41:17,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 09:41:18.165 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.183 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.201 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.217 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.234 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.254 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.325 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.333 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.349 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.361 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.372 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.387 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.398 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.405 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.412 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:18.419 UTC [16371] WARNING: there is no transaction in progress
2026-08-12 09:41:19,120 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 09:41:19,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 09:41:19,375 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 09:41:19,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 09:41:19,623 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 09:41:19,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 09:41:19,890 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 09:41:19,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 09:41:20,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 09:41:20,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cbbz.c.gcp-upstream.internal';"
2026-08-12 09:41:20,631 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 09:41:20,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 09:41:20,894 [I] DB_PATCHES have already been applied
I20260812 09:41:20.900625 3660 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknDSHUD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 09:41:20.900692 3660 mini_ranger.cc:193] Using host: 127.3.147.20
I20260812 09:41:20.904850 3660 mini_ranger.cc:227] Ranger admin URL: http://127.3.147.20:33405
Aug 12, 2026 9:41:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:41:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:41:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 9:41:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 9:41:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 9:41:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 9:41:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 09:41:22.068600 3660 mini_ranger.cc:160] Time spent starting Ranger: real 8.959s user 0.003s sys 0.009s
Aug 12, 2026 9:41:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 9:41:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 9:41:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 9:41:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 9:41:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 09:41:44.932185 3660 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.147.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.147.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/rangerkms_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/rangerkms_127.3.147.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.147.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.147.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.147.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/HTTP_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.147.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/HTTP_127.3.147.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 09:41:45 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527705, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 09:41:45.030092 3660 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 09:41:45.043502 3660 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 09:41:45.950600 3660 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 09:41:45.950706 3660 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms
I20260812 09:41:45.950821 3660 mini_ranger_kms.cc:227] Using postgres at 127.3.147.20:36571
2026-08-12 09:41:46,853 [I] DB FLAVOR :POSTGRES
2026-08-12 09:41:46,853 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 09:41:46,853 [I] Checking connection
2026-08-12 09:41:46,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 09:41:47,197 [I] connection success
2026-08-12 09:41:47,197 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 09:41:47,197 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 09:41:47,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 09:41:47,443 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 09:41:47,443 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 09:41:47,443 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 09:41:47,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:36571/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 09:41:47,729 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 09:41:48.248696 3660 mini_ranger_kms.cc:310] Created kms service
I20260812 09:41:48.433584 3660 mini_ranger_kms.cc:326] Created kudu user
I20260812 09:41:48.481737 3660 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 09:41:48 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527705, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:41:48.922134 3660 mini_ranger_kms.cc:384] Added ranger policy
I20260812 09:41:48.922312 3660 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 09:41:48.922348 3660 mini_ranger_kms.cc:240] Using host: 127.3.147.20
I20260812 09:41:48.926923 3660 mini_ranger_kms.cc:276] Ranger KMS PID: 16579
I20260812 09:41:48.927071 3660 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.147.20:54065
Aug 12, 2026 9:41:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:41:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:41:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 9:41:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 9:41:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 9:41:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 09:41:51.286513 3660 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.256s user 0.028s sys 0.452s
I20260812 09:41:51.286674 3660 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 09:41:51.286825 3660 mini_ranger_kms.cc:401] 127.3.147.20:54065/kms/v1/keys
Aug 12 09:41:51 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786527711, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.147.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 09:41:51 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527711, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.147.20@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
Aug 12 09:41:52 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527711, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.147.20@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
Aug 12 09:41:54 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:41:54.805266 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43447
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:43447
--ranger_config_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:54065/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:41:54.924837 16638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:54.925130 16638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:54.925209 16638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:54.929126 16638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:41:54.929203 16638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:54.929241 16638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:54.929297 16638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:41:54.929337 16638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:41:54.934615 16638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.147.20:54065/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:43447
--ranger_config_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:54.935631 16638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:54.936692 16638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:54.945681 16644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:54.945734 16643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:54.946043 16638 server_base.cc:1034] running on GCE node
W20260812 09:41:54.945700 16646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:54.946753 16638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:54.947819 16638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:54.949005 16638 hybrid_clock.cc:648] HybridClock initialized: now 1786527714948972 us; error 51 us; skew 500 ppm
Aug 12 09:41:54 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527714, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:41:54.955248 16638 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:41:54.956285 16638 webserver.cc:460] Webserver started at http://127.3.147.62:46717/ using document root <none> and password file <none>
I20260812 09:41:54.956904 16638 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:54.956960 16638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:54.957188 16638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 09:41:54 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527714, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:41:55.020968 16638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "24271f17f2d144199ff9df62a1f03d84"
format_stamp: "Formatted at 2026-08-12 09:41:55 on dist-test-slave-cbbz"
server_key: "75409b14e88827c998fe531fc46db7bb"
server_key_iv: "310565a47443b51d3b52473b89e97557"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:55.021715 16638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "24271f17f2d144199ff9df62a1f03d84"
format_stamp: "Formatted at 2026-08-12 09:41:55 on dist-test-slave-cbbz"
server_key: "75409b14e88827c998fe531fc46db7bb"
server_key_iv: "310565a47443b51d3b52473b89e97557"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:55.026513 16638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:41:55.045149 16653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:55.046676 16638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:41:55.046834 16638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "24271f17f2d144199ff9df62a1f03d84"
format_stamp: "Formatted at 2026-08-12 09:41:55 on dist-test-slave-cbbz"
server_key: "75409b14e88827c998fe531fc46db7bb"
server_key_iv: "310565a47443b51d3b52473b89e97557"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:55.046984 16638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:55.066040 16638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:55.069237 16638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:55.069443 16638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:55.078140 16638 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:43447
I20260812 09:41:55.078186 16711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:43447 every 8 connection(s)
I20260812 09:41:55.079309 16638 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:41:55.082217 16712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:55.083874 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 16638
I20260812 09:41:55.084009 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:41:55.084370 3660 external_mini_cluster.cc:1428] Setting key 5f6ab13ec2a20de3b2d47935ee479d91
I20260812 09:41:55.089412 16712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84: Bootstrap starting.
I20260812 09:41:55.092142 16712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:55.093154 16712 log.cc:824] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:55.095207 16712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84: No bootstrap required, opened a new log
Aug 12 09:41:55 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:41:55.099543 16712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24271f17f2d144199ff9df62a1f03d84" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43447 } }
I20260812 09:41:55.099790 16712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:55.099890 16712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24271f17f2d144199ff9df62a1f03d84, State: Initialized, Role: FOLLOWER
I20260812 09:41:55.100419 16712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24271f17f2d144199ff9df62a1f03d84" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43447 } }
I20260812 09:41:55.100668 16712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:41:55.100764 16712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:41:55.100894 16712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:55.101840 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:55.085895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.3.147.62:43447)
0812 09:41:55.086467 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:55.086489 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:55.086532 (+ 43us) server_negotiation.cc:407] Connection header received
0812 09:41:55.087426 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:55.087451 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:55.087888 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:55.088339 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:55.089246 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:55.090738 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:55.091964 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:55.092751 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:55.092946 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:55.096113 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:55.096121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:55.096161 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:55.096194 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:55.098313 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:55.098870 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:55.098876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:55.098882 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:55.099009 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:55.099351 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:55.099355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:55.099357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:55.099836 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:55.100026 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:55.100471 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 09:41:55.100914 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":159,"threads_started":1}
I20260812 09:41:55.101929 16712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24271f17f2d144199ff9df62a1f03d84" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43447 } }
I20260812 09:41:55.102253 16712 leader_election.cc:302] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24271f17f2d144199ff9df62a1f03d84; no voters:
I20260812 09:41:55.102571 16712 leader_election.cc:288] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:41:55.102774 16717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:41:55.103108 16717 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [term 1 LEADER]: Becoming Leader. State: Replica: 24271f17f2d144199ff9df62a1f03d84, State: Running, Role: LEADER
I20260812 09:41:55.103461 16717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24271f17f2d144199ff9df62a1f03d84" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43447 } }
I20260812 09:41:55.103842 16712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:41:55.104768 16717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24271f17f2d144199ff9df62a1f03d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24271f17f2d144199ff9df62a1f03d84" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43447 } } }
I20260812 09:41:55.104940 16717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [sys.catalog]: This master's current role is: LEADER
I20260812 09:41:55.105320 16717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24271f17f2d144199ff9df62a1f03d84. Latest consensus state: current_term: 1 leader_uuid: "24271f17f2d144199ff9df62a1f03d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24271f17f2d144199ff9df62a1f03d84" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43447 } } }
I20260812 09:41:55.105412 16717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84 [sys.catalog]: This master's current role is: LEADER
I20260812 09:41:55.106272 16727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:41:55.109551 16727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:41:55.110112 16712 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 09:41:55.120837 16727 catalog_manager.cc:1347] Generated new cluster ID: 546907123a6947c6bdde69906e47152f
I20260812 09:41:55.120990 16727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:41:55.144750 16727 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:41:55.146955 16727 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:41:55.156389 16727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84: Generated new TSK 0
I20260812 09:41:55.157346 16727 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:41:55.895923 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:54065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43447
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:41:56.041329 16762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:56.041682 16762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:56.041774 16762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:56.048043 16762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:56.048159 16762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:56.048267 16762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:41:56.056382 16762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.147.20:54065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:56.057825 16762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:56.059269 16762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:56.072072 16772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:56.072310 16769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:56.072610 16770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:56.073185 16762 server_base.cc:1034] running on GCE node
I20260812 09:41:56.073626 16762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:56.074462 16762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:56.075665 16762 hybrid_clock.cc:648] HybridClock initialized: now 1786527716075642 us; error 37 us; skew 500 ppm
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:41:56.079470 16762 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:41:56.080554 16762 webserver.cc:460] Webserver started at http://127.3.147.1:43513/ using document root <none> and password file <none>
I20260812 09:41:56.081214 16762 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:56.081305 16762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:56.081518 16762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:41:56.102312 16762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "5a58b3b8d69a4763bbc21f67db566a75"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "8f5e644a276dd314d9eaddaea4aa9a57"
server_key_iv: "241e9c4635a8b623b4c059f06c1397b6"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.103085 16762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "5a58b3b8d69a4763bbc21f67db566a75"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "8f5e644a276dd314d9eaddaea4aa9a57"
server_key_iv: "241e9c4635a8b623b4c059f06c1397b6"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.109014 16762 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 09:41:56.135061 16783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.137465 16762 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 09:41:56.137687 16762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "5a58b3b8d69a4763bbc21f67db566a75"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "8f5e644a276dd314d9eaddaea4aa9a57"
server_key_iv: "241e9c4635a8b623b4c059f06c1397b6"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.137863 16762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:56.159215 16762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:56.162863 16762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:56.163127 16762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:56.163861 16762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:56.165059 16762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:56.165194 16762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.165321 16762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:56.165398 16762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.178604 16762 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:37019
I20260812 09:41:56.179855 16762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:41:56.182894 16908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:37019 every 8 connection(s)
I20260812 09:41:56.185262 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 16762
I20260812 09:41:56.185405 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:41:56.185799 3660 external_mini_cluster.cc:1428] Setting key a5744e600d47f93ef3c0f7848e80b07d
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:41:56.198786 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:56.184349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:57453 (local address 127.3.147.62:43447)
0812 09:41:56.184909 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:56.184914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:56.185363 (+ 449us) server_negotiation.cc:407] Connection header received
0812 09:41:56.186386 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:56.186390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:56.186453 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:56.186519 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:56.188031 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.188986 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.190766 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.191146 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.191224 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:56.194270 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:56.194274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:56.194277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:56.194302 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:56.196276 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:56.196909 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:56.196914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:56.196916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:56.197010 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:56.197442 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:56.197445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:56.197447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:56.197677 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:56.197765 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:56.198458 (+ 693us) server_negotiation.cc:299] Negotiation successful
0812 09:41:56.198589 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":75,"threads_started":1}
I20260812 09:41:56.200498 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:56.184051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43447 (local address 127.3.147.1:57453)
0812 09:41:56.185149 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:56.185250 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:56.186124 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:56.186941 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:56.186950 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:56.187313 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:56.187883 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:56.187898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.189175 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.189179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:56.190625 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:56.190637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.191441 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.191445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:56.191641 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:56.192334 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:56.192355 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:56.194081 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:56.196424 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:56.196430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:56.196446 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:56.196786 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:56.197144 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:56.197147 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:56.197149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:56.197324 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:56.197855 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:56.197861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:56.198183 (+ 322us) client_negotiation.cc:769] Sending connection context
0812 09:41:56.199030 (+ 847us) client_negotiation.cc:240] Negotiation successful
0812 09:41:56.199417 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"thread_start_us":150,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:41:56.202085 16909 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43447
I20260812 09:41:56.202322 16909 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:56.202879 16909 heartbeater.cc:511] Master 127.3.147.62:43447 requested a full tablet report, sending...
I20260812 09:41:56.204753 16671 ts_manager.cc:194] Registered new tserver with Master: 5a58b3b8d69a4763bbc21f67db566a75 (127.3.147.1:37019)
I20260812 09:41:56.206317 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:57453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:41:56.219388 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:54065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43447
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 09:41:56.384680 16917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:56.385061 16917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:56.385130 16917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:56.391232 16917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:56.391315 16917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:56.391405 16917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:41:56.399278 16917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.147.20:54065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:56.400647 16917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:56.402048 16917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:56.414652 16925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:56.414637 16928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:56.415267 16926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:56.415807 16917 server_base.cc:1034] running on GCE node
I20260812 09:41:56.416312 16917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:56.417104 16917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:56.418311 16917 hybrid_clock.cc:648] HybridClock initialized: now 1786527716418288 us; error 39 us; skew 500 ppm
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:41:56.421994 16917 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:41:56.423130 16917 webserver.cc:460] Webserver started at http://127.3.147.2:32791/ using document root <none> and password file <none>
I20260812 09:41:56.423874 16917 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:56.423988 16917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:56.424319 16917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:41:56.443956 16917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "9f90eea041304e219a677e59c57e7f56"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "0cf5aa4a08a1fbf6a5d521e1a73f2523"
server_key_iv: "2a21f1e1c5a66f12f804deda4bde4f22"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.444784 16917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "9f90eea041304e219a677e59c57e7f56"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "0cf5aa4a08a1fbf6a5d521e1a73f2523"
server_key_iv: "2a21f1e1c5a66f12f804deda4bde4f22"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.450160 16917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 09:41:56.484562 16936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.486524 16917 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.002s sys 0.000s
I20260812 09:41:56.486694 16917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "9f90eea041304e219a677e59c57e7f56"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "0cf5aa4a08a1fbf6a5d521e1a73f2523"
server_key_iv: "2a21f1e1c5a66f12f804deda4bde4f22"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.486868 16917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:56.512840 16917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:56.516022 16917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:56.516268 16917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:56.517083 16917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:56.518062 16917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:56.518134 16917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.518203 16917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:56.518258 16917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.530190 16917 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:41801
I20260812 09:41:56.531154 16917 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:41:56.536404 17059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:41801 every 8 connection(s)
I20260812 09:41:56.541348 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 16917
I20260812 09:41:56.541483 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:41:56.541855 3660 external_mini_cluster.cc:1428] Setting key 26df8060228bd1dc8fff0bcb8d150f09
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:41:56.548187 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:56.533234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:33273 (local address 127.3.147.62:43447)
0812 09:41:56.533405 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:56.533410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:56.534307 (+ 897us) server_negotiation.cc:407] Connection header received
0812 09:41:56.535373 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:56.535377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:56.535437 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:56.535518 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:56.536872 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.537742 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.539458 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.539848 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.539943 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:56.543978 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:56.543982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:56.543984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:56.544013 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:56.545826 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:56.546472 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:56.546476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:56.546477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:56.546564 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:56.546917 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:56.546922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:56.546924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:56.547108 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:56.547238 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:56.547873 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 09:41:56.548005 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:41:56.549238 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:56.533650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43447 (local address 127.3.147.2:33273)
0812 09:41:56.534148 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:56.534204 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:56.535156 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:56.535694 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:56.535701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:56.535974 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:56.536666 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:56.536699 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.537918 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.537921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:56.539312 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:56.539323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.540222 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.540228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:56.540419 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:56.541124 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:56.541148 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:56.543786 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:56.545977 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:56.545982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:56.545998 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:56.546348 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:56.546660 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:56.546665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:56.546666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:56.546813 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:56.547238 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:56.547246 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:56.547596 (+ 350us) client_negotiation.cc:769] Sending connection context
0812 09:41:56.547829 (+ 233us) client_negotiation.cc:240] Negotiation successful
0812 09:41:56.548210 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":153,"threads_started":1}
I20260812 09:41:56.550475 17060 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43447
I20260812 09:41:56.550735 17060 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:56.551337 17060 heartbeater.cc:511] Master 127.3.147.62:43447 requested a full tablet report, sending...
I20260812 09:41:56.552431 16671 ts_manager.cc:194] Registered new tserver with Master: 9f90eea041304e219a677e59c57e7f56 (127.3.147.2:41801)
I20260812 09:41:56.553207 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:33273
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:41:56.597903 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:54065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43447
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
W20260812 09:41:56.819517 17065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:41:56.819986 17065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:41:56.820101 17065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:41:56.826731 17065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:41:56.826871 17065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:41:56.827008 17065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:41:56.834043 17065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.147.20:54065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:41:56.835278 17065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:41:56.836424 17065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:41:56.846475 17072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:56.846871 17074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:41:56.848318 17070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:41:56.849709 17065 server_base.cc:1034] running on GCE node
I20260812 09:41:56.850226 17065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:41:56.851076 17065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:41:56.858419 17065 hybrid_clock.cc:648] HybridClock initialized: now 1786527716858345 us; error 91 us; skew 500 ppm
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:41:56.862098 17065 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:41:56.863240 17065 webserver.cc:460] Webserver started at http://127.3.147.3:38379/ using document root <none> and password file <none>
I20260812 09:41:56.864013 17065 fs_manager.cc:359] Metadata directory not provided
I20260812 09:41:56.864188 17065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:41:56.864480 17065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:41:56.886801 17065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "a04e978afef648109b01063a75e36ef6"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "6d55f7b1689fd2922c86b9050df6f49b"
server_key_iv: "72b30c5de7b3028bcb687f8ce338f244"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.888659 17065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "a04e978afef648109b01063a75e36ef6"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "6d55f7b1689fd2922c86b9050df6f49b"
server_key_iv: "72b30c5de7b3028bcb687f8ce338f244"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.894498 17065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 09:41:56.919042 17082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.920930 17065 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260812 09:41:56.921078 17065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "a04e978afef648109b01063a75e36ef6"
format_stamp: "Formatted at 2026-08-12 09:41:56 on dist-test-slave-cbbz"
server_key: "6d55f7b1689fd2922c86b9050df6f49b"
server_key_iv: "72b30c5de7b3028bcb687f8ce338f244"
server_key_version: "kuduclusterkey@0"
I20260812 09:41:56.921216 17065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:41:56.941411 17065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:41:56.945246 17065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:41:56.945581 17065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:41:56.946333 17065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:41:56.947445 17065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:41:56.947552 17065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.947681 17065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:41:56.947757 17065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:41:56.968369 17065 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42247
I20260812 09:41:56.969712 17065 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:41:56.970638 17210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42247 every 8 connection(s)
I20260812 09:41:56.977226 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 17065
I20260812 09:41:56.977362 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:41:56.977677 3660 external_mini_cluster.cc:1428] Setting key 477fdd9b42b5f8b806ac932f27dcdeb1
Aug 12 09:41:56 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:41:56.997622 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:56.974982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:55339 (local address 127.3.147.62:43447)
0812 09:41:56.975121 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:56.975126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:56.980721 (+ 5595us) server_negotiation.cc:407] Connection header received
0812 09:41:56.981613 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:56.981616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:56.981670 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:56.981746 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:56.985499 (+ 3753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.986322 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.988021 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.988393 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.988480 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:56.993094 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:56.993099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:56.993102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:56.993133 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:56.994929 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:56.995568 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:56.995586 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:56.995587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:56.995679 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:56.996116 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:56.996120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:56.996122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:56.996375 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:56.996466 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:56.997323 (+ 857us) server_negotiation.cc:299] Negotiation successful
0812 09:41:56.997444 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 09:41:56.999434 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:56.973809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43447 (local address 127.3.147.3:55339)
0812 09:41:56.980552 (+ 6743us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:56.980599 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:56.981411 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:56.984095 (+ 2684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:56.984105 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:56.984560 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:41:56.985317 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:56.985339 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.986492 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.986496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:56.987873 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:56.987884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:56.988707 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:56.988712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:56.988941 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:56.989688 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:41:56.989711 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:41:56.992895 (+ 3184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:41:56.995063 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:56.995068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:56.995092 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:56.995443 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:56.995805 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:41:56.995808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:41:56.995810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:41:56.995989 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:41:56.996705 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:41:56.996711 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:41:56.997020 (+ 309us) client_negotiation.cc:769] Sending connection context
0812 09:41:56.998061 (+ 1041us) client_negotiation.cc:240] Negotiation successful
0812 09:41:56.998380 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6547,"thread_start_us":143,"threads_started":1}
I20260812 09:41:57.000926 17211 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43447
I20260812 09:41:57.001159 17211 heartbeater.cc:464] Registering TS with master...
I20260812 09:41:57.001752 17211 heartbeater.cc:511] Master 127.3.147.62:43447 requested a full tablet report, sending...
I20260812 09:41:57.002985 16671 ts_manager.cc:194] Registered new tserver with Master: a04e978afef648109b01063a75e36ef6 (127.3.147.3:42247)
I20260812 09:41:57.003738 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:55339
I20260812 09:41:57.007395 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:41:57.020013 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.009706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44528 (local address 127.3.147.62:43447)
0812 09:41:57.009812 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:57.009816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:57.010566 (+ 750us) server_negotiation.cc:407] Connection header received
0812 09:41:57.010745 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:57.010749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:57.010789 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:57.010834 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:41:57.011813 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.012642 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.013849 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.014188 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.014265 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.016051 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:41:57.016056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:41:57.016059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:41:57.016098 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:41:57.017923 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:57.018451 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:57.018455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:57.018456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:57.018547 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:41:57.019099 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:41:57.019107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:41:57.019109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:41:57.019316 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:41:57.019450 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:57.019739 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 09:41:57.019864 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 09:41:57.023672 16671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:41:57.028119 16671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:41:57.050900 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.040891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42247 (local address 127.0.0.1:49156)
0812 09:41:57.044567 (+ 3676us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:57.044631 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:57.044782 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:57.045930 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:57.045936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:57.045960 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:57.046093 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.046099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.047607 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.047610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.049062 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.049072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.050330 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.050334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.050447 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.050496 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 09:41:57.050615 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 09:41:57.050739 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3499,"thread_start_us":107,"threads_started":1}
I20260812 09:41:57.051463 17227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.043660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:41801 (local address 127.0.0.1:33748)
0812 09:41:57.046300 (+ 2640us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:57.046311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:57.046380 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:57.047139 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:57.047142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:57.047159 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:57.047242 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.047248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.048667 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.048671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.049843 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.049854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.051092 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.051096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.051182 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.051204 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 09:41:57.051272 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 09:41:57.051336 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2584,"thread_start_us":94,"threads_started":1}
I20260812 09:41:57.051628 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.041130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.3.147.3:42247)
0812 09:41:57.045297 (+ 4167us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:57.045308 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:57.045330 (+ 22us) server_negotiation.cc:407] Connection header received
0812 09:41:57.045401 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:57.045406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:57.045621 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:57.045766 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:57.046235 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.047451 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.049228 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.050147 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.051053 (+ 906us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.051264 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:57.051323 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 09:41:57.051461 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4067,"thread_start_us":198,"threads_started":1}
I20260812 09:41:57.051855 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.043849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.3.147.2:41801)
0812 09:41:57.046593 (+ 2744us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:57.046621 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:57.046641 (+ 20us) server_negotiation.cc:407] Connection header received
0812 09:41:57.046708 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:57.046712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:57.046876 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:57.047007 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:57.047365 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.048475 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.050007 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.050785 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.050892 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.051028 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:57.051635 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 09:41:57.051733 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2646,"thread_start_us":112,"threads_started":1}
I20260812 09:41:57.053110 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.043300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37019 (local address 127.0.0.1:43874)
0812 09:41:57.044993 (+ 1693us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:57.045004 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:57.045068 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:57.046024 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:57.046026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:57.046042 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:57.046135 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.046140 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.048319 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.048323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.049444 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.049454 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.052728 (+ 3274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.052732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.052820 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.052843 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:41:57.052909 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 09:41:57.052975 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1635,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260812 09:41:57.053755 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.044105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.3.147.1:37019)
0812 09:41:57.045272 (+ 1167us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:57.045281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:57.045306 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:41:57.045388 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:57.045393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:57.045697 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:57.045885 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:57.046300 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.047451 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.049575 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.050713 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.050810 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.050953 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:57.053522 (+ 2569us) server_negotiation.cc:299] Negotiation successful
0812 09:41:57.053621 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1043,"thread_start_us":126,"threads_started":1}
I20260812 09:41:57.054811 17122 tablet_service.cc:1467] Processing CreateTablet for tablet 64ab5d4fc19c44fc9ad8b21c34969299 (DEFAULT_TABLE table=test-table [id=32ceab227df74eaf9df269b883250bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:41:57.054807 16970 tablet_service.cc:1467] Processing CreateTablet for tablet 64ab5d4fc19c44fc9ad8b21c34969299 (DEFAULT_TABLE table=test-table [id=32ceab227df74eaf9df269b883250bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:41:57.055929 17122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ab5d4fc19c44fc9ad8b21c34969299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:57.055930 16970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ab5d4fc19c44fc9ad8b21c34969299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:57.056290 16835 tablet_service.cc:1467] Processing CreateTablet for tablet 64ab5d4fc19c44fc9ad8b21c34969299 (DEFAULT_TABLE table=test-table [id=32ceab227df74eaf9df269b883250bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:41:57.057210 16835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ab5d4fc19c44fc9ad8b21c34969299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:41:57.064504 17235 tablet_bootstrap.cc:492] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: Bootstrap starting.
I20260812 09:41:57.065302 17237 tablet_bootstrap.cc:492] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75: Bootstrap starting.
I20260812 09:41:57.067237 17235 tablet_bootstrap.cc:654] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:57.067688 17237 tablet_bootstrap.cc:654] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:57.068075 17235 log.cc:824] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:57.069047 17237 log.cc:824] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:57.069731 17236 tablet_bootstrap.cc:492] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56: Bootstrap starting.
I20260812 09:41:57.071233 17237 tablet_bootstrap.cc:492] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75: No bootstrap required, opened a new log
I20260812 09:41:57.071480 17237 ts_tablet_manager.cc:1397] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:41:57.075817 17236 tablet_bootstrap.cc:654] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56: Neither blocks nor log segments found. Creating new log.
I20260812 09:41:57.077083 17235 tablet_bootstrap.cc:492] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: No bootstrap required, opened a new log
I20260812 09:41:57.077284 17235 ts_tablet_manager.cc:1397] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260812 09:41:57.079180 17237 raft_consensus.cc:348] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.080410 17236 log.cc:824] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56: Log is configured to *not* fsync() on all Append() calls
I20260812 09:41:57.080966 17237 raft_consensus.cc:374] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:57.081115 17237 raft_consensus.cc:729] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a58b3b8d69a4763bbc21f67db566a75, State: Initialized, Role: FOLLOWER
I20260812 09:41:57.081670 17235 raft_consensus.cc:348] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.082002 17235 raft_consensus.cc:374] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:57.082091 17235 raft_consensus.cc:729] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a04e978afef648109b01063a75e36ef6, State: Initialized, Role: FOLLOWER
I20260812 09:41:57.082274 17237 consensus_queue.cc:260] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.082566 17235 consensus_queue.cc:260] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.083560 17235 ts_tablet_manager.cc:1428] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:41:57.083600 17237 ts_tablet_manager.cc:1428] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260812 09:41:57.083956 16909 heartbeater.cc:503] Master 127.3.147.62:43447 was elected leader, sending a full tablet report...
I20260812 09:41:57.083956 17211 heartbeater.cc:503] Master 127.3.147.62:43447 was elected leader, sending a full tablet report...
I20260812 09:41:57.085767 17236 tablet_bootstrap.cc:492] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56: No bootstrap required, opened a new log
I20260812 09:41:57.085947 17236 ts_tablet_manager.cc:1397] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260812 09:41:57.094165 17236 raft_consensus.cc:348] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.094580 17236 raft_consensus.cc:374] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:41:57.094686 17236 raft_consensus.cc:729] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f90eea041304e219a677e59c57e7f56, State: Initialized, Role: FOLLOWER
I20260812 09:41:57.095461 17236 consensus_queue.cc:260] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.097143 17236 ts_tablet_manager.cc:1428] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
Aug 12 09:41:57 dist-test-slave-cbbz krb5kdc[16134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527716, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:41:57.101007 17060 heartbeater.cc:503] Master 127.3.147.62:43447 was elected leader, sending a full tablet report...
I20260812 09:41:57.123025 17243 raft_consensus.cc:482] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:41:57.123288 17243 raft_consensus.cc:504] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.124673 17243 leader_election.cc:288] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a58b3b8d69a4763bbc21f67db566a75 (127.3.147.1:37019), 9f90eea041304e219a677e59c57e7f56 (127.3.147.2:41801)
I20260812 09:41:57.133281 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.128416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37019 (local address 127.3.147.3:33747)
0812 09:41:57.128938 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:57.128953 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:57.129052 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:57.129408 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:57.129412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:57.129431 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:57.129552 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.129558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.130603 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.130606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.131769 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.131781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.132786 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.132790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.132911 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.132948 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 09:41:57.133052 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 09:41:57.133145 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456}
I20260812 09:41:57.133857 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.128554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:33747 (local address 127.3.147.1:37019)
0812 09:41:57.128702 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:57.128707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:57.129042 (+ 335us) server_negotiation.cc:407] Connection header received
0812 09:41:57.129151 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:57.129154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:57.129208 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:57.129289 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:57.129691 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.130462 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.131913 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.132525 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.132639 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.132705 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:57.133662 (+ 957us) server_negotiation.cc:299] Negotiation successful
0812 09:41:57.133738 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 09:41:57.134613 16858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ab5d4fc19c44fc9ad8b21c34969299" candidate_uuid: "a04e978afef648109b01063a75e36ef6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a58b3b8d69a4763bbc21f67db566a75" is_pre_election: true
I20260812 09:41:57.134982 16858 raft_consensus.cc:2454] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a04e978afef648109b01063a75e36ef6 in term 0.
I20260812 09:41:57.135594 17083 leader_election.cc:302] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a58b3b8d69a4763bbc21f67db566a75, a04e978afef648109b01063a75e36ef6; no voters:
I20260812 09:41:57.135962 17243 raft_consensus.cc:2781] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:41:57.136118 17243 raft_consensus.cc:482] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:41:57.136246 17243 raft_consensus.cc:3037] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:57.138013 17243 raft_consensus.cc:504] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.138728 16858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ab5d4fc19c44fc9ad8b21c34969299" candidate_uuid: "a04e978afef648109b01063a75e36ef6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a58b3b8d69a4763bbc21f67db566a75"
I20260812 09:41:57.138854 16858 raft_consensus.cc:3037] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:57.140113 16858 raft_consensus.cc:2454] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a04e978afef648109b01063a75e36ef6 in term 1.
I20260812 09:41:57.140517 17083 leader_election.cc:302] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a58b3b8d69a4763bbc21f67db566a75, a04e978afef648109b01063a75e36ef6; no voters:
I20260812 09:41:57.140974 17243 leader_election.cc:288] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [CANDIDATE]: Term 1 election: Requested vote from peers 5a58b3b8d69a4763bbc21f67db566a75 (127.3.147.1:37019), 9f90eea041304e219a677e59c57e7f56 (127.3.147.2:41801)
I20260812 09:41:57.141129 17243 raft_consensus.cc:2781] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:41:57.141242 17243 raft_consensus.cc:686] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 1 LEADER]: Becoming Leader. State: Replica: a04e978afef648109b01063a75e36ef6, State: Running, Role: LEADER
I20260812 09:41:57.141641 17243 consensus_queue.cc:237] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } }
I20260812 09:41:57.145260 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.129314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:52715 (local address 127.3.147.2:41801)
0812 09:41:57.129422 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:57.129426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:57.132233 (+ 2807us) server_negotiation.cc:407] Connection header received
0812 09:41:57.132311 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:57.132314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:57.132370 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:57.132440 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:41:57.133350 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.134054 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.137630 (+ 3576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.138037 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.138165 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.138234 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:57.144977 (+ 6743us) server_negotiation.cc:299] Negotiation successful
0812 09:41:57.145092 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:41:57.145921 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.128663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:41801 (local address 127.3.147.3:52715)
0812 09:41:57.131489 (+ 2826us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:57.131503 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:41:57.131592 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:41:57.132705 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:41:57.132708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:41:57.132725 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:41:57.132877 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.132883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.136295 (+ 3412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.136298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.137369 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:41:57.137383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.138198 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.138202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:41:57.138299 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.138324 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 09:41:57.145668 (+ 7344us) client_negotiation.cc:240] Negotiation successful
0812 09:41:57.145762 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2757,"thread_start_us":105,"threads_started":1}
I20260812 09:41:57.146170 16671 catalog_manager.cc:5495] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a04e978afef648109b01063a75e36ef6 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "a04e978afef648109b01063a75e36ef6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04e978afef648109b01063a75e36ef6" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 } health_report { overall_health: UNKNOWN } } }
I20260812 09:41:57.146917 17007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ab5d4fc19c44fc9ad8b21c34969299" candidate_uuid: "a04e978afef648109b01063a75e36ef6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f90eea041304e219a677e59c57e7f56" is_pre_election: true
I20260812 09:41:57.147240 17007 raft_consensus.cc:2454] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a04e978afef648109b01063a75e36ef6 in term 0.
I20260812 09:41:57.147611 17009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ab5d4fc19c44fc9ad8b21c34969299" candidate_uuid: "a04e978afef648109b01063a75e36ef6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f90eea041304e219a677e59c57e7f56"
I20260812 09:41:57.147719 17009 raft_consensus.cc:3037] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:41:57.149173 17009 raft_consensus.cc:2454] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a04e978afef648109b01063a75e36ef6 in term 1.
I20260812 09:41:57.181265 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.174439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.3.147.3:42247)
0812 09:41:57.174890 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 09:41:57.174895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:41:57.174909 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:41:57.175011 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:41:57.175014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:41:57.175085 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:41:57.175200 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:41:57.175581 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.176456 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.177653 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:41:57.178017 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:41:57.178130 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:41:57.179847 (+ 1717us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:41:57.180410 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:41:57.180550 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 09:41:57.181034 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 09:41:57.181107 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363}
W20260812 09:41:57.182770 16911 tablet.cc:2369] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:41:57.188097 17009 raft_consensus.cc:1261] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 1 FOLLOWER]: Refusing update from remote peer a04e978afef648109b01063a75e36ef6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:41:57.188163 16858 raft_consensus.cc:1261] T 64ab5d4fc19c44fc9ad8b21c34969299 P 5a58b3b8d69a4763bbc21f67db566a75 [term 1 FOLLOWER]: Refusing update from remote peer a04e978afef648109b01063a75e36ef6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:41:57.188876 17246 consensus_queue.cc:1035] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f90eea041304e219a677e59c57e7f56" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 41801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:41:57.189266 17246 consensus_queue.cc:1035] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a58b3b8d69a4763bbc21f67db566a75" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:41:57.201978 17250 mvcc.cc:205] Tried to move back new op lower bound from 7317617529594769408 to 7317617529418133504. Current Snapshot: MvccSnapshot[applied={T|T < 7317617529594769408}]
I20260812 09:41:57.209072 17252 mvcc.cc:205] Tried to move back new op lower bound from 7317617529594769408 to 7317617529418133504. Current Snapshot: MvccSnapshot[applied={T|T < 7317617529594769408}]
I20260812 09:41:57.216893 17251 mvcc.cc:205] Tried to move back new op lower bound from 7317617529594769408 to 7317617529418133504. Current Snapshot: MvccSnapshot[applied={T|T < 7317617529594769408}]
I20260812 09:41:57.220216 16670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:41:57.220381 16670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:41:57.223232 16670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24271f17f2d144199ff9df62a1f03d84: Sending DeleteTablet for 3 replicas of tablet 64ab5d4fc19c44fc9ad8b21c34969299
I20260812 09:41:57.224457 17122 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ab5d4fc19c44fc9ad8b21c34969299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:41:57 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:49156
I20260812 09:41:57.224960 16835 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ab5d4fc19c44fc9ad8b21c34969299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:41:57 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:43874
I20260812 09:41:57.226017 16970 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ab5d4fc19c44fc9ad8b21c34969299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:41:57 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:33748
I20260812 09:41:57.227384 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 16762
I20260812 09:41:57.237355 17261 tablet_replica.cc:330] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: stopping tablet replica
I20260812 09:41:57.244267 17263 tablet_replica.cc:330] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56: stopping tablet replica
I20260812 09:41:57.244755 17261 raft_consensus.cc:2229] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:41:57.245124 17263 raft_consensus.cc:2229] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:41:57.245546 17263 raft_consensus.cc:2258] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:41:57.245988 17261 raft_consensus.cc:2258] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:41:57.247800 17261 ts_tablet_manager.cc:1905] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:41:57.250893 17261 ts_tablet_manager.cc:1918] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:41:57.251080 17261 log.cc:1196] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/64ab5d4fc19c44fc9ad8b21c34969299
I20260812 09:41:57.251742 17261 ts_tablet_manager.cc:1939] T 64ab5d4fc19c44fc9ad8b21c34969299 P a04e978afef648109b01063a75e36ef6: Deleting consensus metadata
I20260812 09:41:57.253407 17263 ts_tablet_manager.cc:1905] T 64ab5d4fc19c44fc9ad8b21c34969299 P 9f90eea041304e219a677e59c57e7f56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:41:57.253631 16655 catalog_manager.cc:4841] TS a04e978afef648109b01063a75e36ef6 (127.3.147.3:42247): tablet 64ab5d4fc19c44fc9ad8b21c34969299 (table test-table [id=32ceab227df74eaf9df269b883250bd7]) successfully deleted
W20260812 09:41:57.254171 16654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:37019 (error 108)
I20260812 09:41:57.255156 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 16917
I20260812 09:41:57.268325 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 17065
I20260812 09:41:57.270594 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.270095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37019 (local address 127.0.0.1:43880)
0812 09:41:57.270360 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:57.270465 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260812 09:41:57.270934 16654 catalog_manager.cc:4584] TS 5a58b3b8d69a4763bbc21f67db566a75 (127.3.147.1:37019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64ab5d4fc19c44fc9ad8b21c34969299: Network error: Client connection negotiation failed: client connection to 127.3.147.1:37019: connect: Connection refused (error 111)
I20260812 09:41:57.271790 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:41:57.271494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:41801 (local address 127.0.0.1:33752)
0812 09:41:57.271656 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 09:41:57.271710 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.2:41801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 09:41:57.281005 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 16638
I20260812 09:41:57.290233 3660 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 09:41:57.634558 3660 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 09:41:57.634721 3660 mini_ranger.cc:66] Stopping Ranger...
I20260812 09:41:57.980245 3660 mini_ranger.cc:68] Stopped Ranger
2026-08-12 09:41:57.980 UTC [16168] LOG: received smart shutdown request
2026-08-12 09:41:57.984 UTC [16168] LOG: background worker "logical replication launcher" (PID 16180) exited with exit code 1
2026-08-12 09:41:57.985 UTC [16173] LOG: shutting down
2026-08-12 09:41:58.162 UTC [16168] LOG: database system is shut down
2026-08-12T09:41:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50109 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:41:58 dist-test-slave-cbbz krb5kdc[17273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:41:58 dist-test-slave-cbbz krb5kdc[17273](info): set up 2 sockets
Aug 12 09:41:58 dist-test-slave-cbbz krb5kdc[17273](info): commencing operation
krb5kdc: starting...
W20260812 09:41:59.409226 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:41:59 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:41:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:41:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.147.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.147.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/rangeradmin_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/rangeradmin_127.3.147.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.147.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.147.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/rangerlookup_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/rangerlookup_127.3.147.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.147.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.147.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/HTTP_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/HTTP_127.3.147.20@KRBTEST.COM.keytab.
I20260812 09:41:59.580478 3660 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknDSHUD/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/postgres -l logfile start
2026-08-12 09:42:02.123 UTC [17305] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 09:42:02.123 UTC [17305] LOG: listening on IPv4 address "127.3.147.20", port 37445
2026-08-12 09:42:02.126 UTC [17305] LOG: listening on Unix socket "/tmp/.s.PGSQL.37445"
2026-08-12 09:42:02.148 UTC [17309] LOG: database system was shut down at 2026-08-12 09:42:00 UTC
2026-08-12 09:42:02.152 UTC [17305] LOG: database system is ready to accept connections
I20260812 09:42:03.176358 3660 mini_postgres.cc:96] Postgres bound to 37445
2026-08-12 09:42:03.183 UTC [17320] FATAL: database "slave" does not exist
127.3.147.20:37445 - accepting connections
I20260812 09:42:03.184075 3660 mini_ranger.cc:161] Starting Ranger...
I20260812 09:42:03.199469 3660 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 09:42:03.686403 3660 mini_ranger.cc:87] Created ranger Postgres database
I20260812 09:42:03.686506 3660 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-admin
2026-08-12 09:42:03,978 [I] DB FLAVOR :POSTGRES
2026-08-12 09:42:03,978 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 09:42:03,978 [I] Checking connection..
2026-08-12 09:42:03,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 09:42:04,251 [I] Checking connection passed.
2026-08-12 09:42:04,252 [I] --------- Verifying version history table ---------
2026-08-12 09:42:04,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 09:42:04,518 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 09:42:04,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 09:42:04,796 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 09:42:04,796 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 09:42:04,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 09:42:05.103 UTC [17393] WARNING: there is no transaction in progress
2026-08-12 09:42:05,111 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 09:42:05,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 09:42:05,358 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 09:42:05,358 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 09:42:05,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 09:42:05,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 09:42:05,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cbbz.c.gcp-upstream.internal','N') ;"
2026-08-12 09:42:06,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 09:42:06,393 [I] Table x_portal_user does not exist in database ranger
2026-08-12 09:42:06,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 09:42:06,682 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 09:42:06,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 09:42:06,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 09:42:07,164 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 09:42:07,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 09:42:08.133 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.148 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.163 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.179 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.195 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.213 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.270 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.277 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.292 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.303 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.314 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.328 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.338 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.344 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.351 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:08.358 UTC [17510] WARNING: there is no transaction in progress
2026-08-12 09:42:09,025 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 09:42:09,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 09:42:09,284 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 09:42:09,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 09:42:09,552 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 09:42:09,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 09:42:09,819 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 09:42:09,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 09:42:10,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 09:42:10,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cbbz.c.gcp-upstream.internal';"
2026-08-12 09:42:10,557 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 09:42:10,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 09:42:10,808 [I] DB_PATCHES have already been applied
I20260812 09:42:10.814765 3660 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknDSHUD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 09:42:10.814841 3660 mini_ranger.cc:193] Using host: 127.3.147.20
I20260812 09:42:10.819109 3660 mini_ranger.cc:227] Ranger admin URL: http://127.3.147.20:60951
Aug 12, 2026 9:42:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:42:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:42:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 9:42:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 9:42:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 9:42:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 9:42:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 09:42:12.018728 3660 mini_ranger.cc:160] Time spent starting Ranger: real 8.835s user 0.000s sys 0.011s
Aug 12, 2026 9:42:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 9:42:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 9:42:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 9:42:31 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 9:42:32 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 09:42:33.281289 3660 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.147.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.147.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/rangerkms_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.147.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/rangerkms_127.3.147.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.147.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.147.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.147.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/HTTP_127.3.147.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.147.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/HTTP_127.3.147.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 09:42:33 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527753, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 09:42:33.368036 3660 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 09:42:33.377791 3660 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 09:42:34.179419 3660 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 09:42:34.179507 3660 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms
I20260812 09:42:34.179526 3660 mini_ranger_kms.cc:227] Using postgres at 127.3.147.20:37445
2026-08-12 09:42:34,381 [I] DB FLAVOR :POSTGRES
2026-08-12 09:42:34,381 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 09:42:34,381 [I] Checking connection
2026-08-12 09:42:34,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 09:42:34,690 [I] connection success
2026-08-12 09:42:34,690 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 09:42:34,690 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 09:42:34,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 09:42:34,948 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 09:42:34,948 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 09:42:34,948 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 09:42:34,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.147.20:37445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 09:42:35,222 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 09:42:35.741856 3660 mini_ranger_kms.cc:310] Created kms service
I20260812 09:42:35.918545 3660 mini_ranger_kms.cc:326] Created kudu user
I20260812 09:42:35.962549 3660 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 09:42:35 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527753, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:36.364008 3660 mini_ranger_kms.cc:384] Added ranger policy
I20260812 09:42:36.364310 3660 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknDSHUD/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknDSHUD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 09:42:36.364347 3660 mini_ranger_kms.cc:240] Using host: 127.3.147.20
I20260812 09:42:36.368738 3660 mini_ranger_kms.cc:276] Ranger KMS PID: 17705
I20260812 09:42:36.368824 3660 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.147.20:35607
Aug 12, 2026 9:42:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:42:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:42:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 9:42:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 9:42:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 9:42:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 09:42:39 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786527759, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.147.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:39.189204 3660 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.821s user 0.013s sys 0.173s
I20260812 09:42:39.189440 3660 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 09:42:39.189683 3660 mini_ranger_kms.cc:401] 127.3.147.20:35607/kms/v1/keys
Aug 12 09:42:40 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527759, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.147.20@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
Aug 12 09:42:40 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527759, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.147.20@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
Aug 12 09:42:42 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:42:42.951452 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35309
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:35309
--ranger_config_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:35607/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:43.064934 17761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:43.065203 17761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:43.065244 17761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:43.069314 17761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:43.069375 17761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:43.069394 17761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:43.069410 17761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:43.069427 17761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:43.075381 17761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.147.20:35607/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:35309
--ranger_config_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:43.076529 17761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:43.077581 17761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:43.086273 17768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:43.086257 17770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:43.086274 17767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:43.086424 17761 server_base.cc:1034] running on GCE node
I20260812 09:42:43.087167 17761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:43.088390 17761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:43.089572 17761 hybrid_clock.cc:648] HybridClock initialized: now 1786527763089559 us; error 38 us; skew 500 ppm
Aug 12 09:42:43 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527763, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:43.096661 17761 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:42:43.097790 17761 webserver.cc:460] Webserver started at http://127.3.147.62:40173/ using document root <none> and password file <none>
I20260812 09:42:43.098409 17761 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:43.098469 17761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:43.098709 17761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 09:42:43 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527763, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:43.159322 17761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "60e5b015702e475c8f224342b163ad79"
format_stamp: "Formatted at 2026-08-12 09:42:43 on dist-test-slave-cbbz"
server_key: "1d9ec07a0b2286d5aae6b83963ef0ca8"
server_key_iv: "24d666f480e5598204791ddb08c12e51"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:43.160086 17761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "60e5b015702e475c8f224342b163ad79"
format_stamp: "Formatted at 2026-08-12 09:42:43 on dist-test-slave-cbbz"
server_key: "1d9ec07a0b2286d5aae6b83963ef0ca8"
server_key_iv: "24d666f480e5598204791ddb08c12e51"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:43.164996 17761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 09:42:43.183743 17779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:43.185191 17761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:42:43.185357 17761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "60e5b015702e475c8f224342b163ad79"
format_stamp: "Formatted at 2026-08-12 09:42:43 on dist-test-slave-cbbz"
server_key: "1d9ec07a0b2286d5aae6b83963ef0ca8"
server_key_iv: "24d666f480e5598204791ddb08c12e51"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:43.185506 17761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:43.233752 17761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:43.236555 17761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:43.236768 17761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:43.244383 17761 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:35309
I20260812 09:42:43.244390 17837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:35309 every 8 connection(s)
I20260812 09:42:43.245369 17761 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:42:43.248124 17838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:43.250578 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 17761
I20260812 09:42:43.250701 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:42:43.251022 3660 external_mini_cluster.cc:1428] Setting key 37b4ea502108acff80cc921349c52682
I20260812 09:42:43.254534 17838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Bootstrap starting.
I20260812 09:42:43.257151 17838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:43.258013 17838 log.cc:824] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:43.260349 17838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: No bootstrap required, opened a new log
Aug 12 09:42:43 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:43.263548 17838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } }
I20260812 09:42:43.263753 17838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:43.263813 17838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60e5b015702e475c8f224342b163ad79, State: Initialized, Role: FOLLOWER
I20260812 09:42:43.264281 17838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } }
I20260812 09:42:43.264446 17838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:42:43.264526 17838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:42:43.264643 17838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:43.265707 17838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } }
I20260812 09:42:43.265928 17842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:43.252524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.3.147.62:35309)
0812 09:42:43.252965 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:43.252980 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:43.253015 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:42:43.253657 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:43.253673 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:43.254003 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:43.254292 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:43.255201 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:43.256225 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:43.257427 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:43.257928 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:43.258044 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:43.261040 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:43.261045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:43.261058 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:43.261082 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:43.262733 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:43.263180 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:43.263191 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:43.263195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:43.263298 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:43.263597 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:43.263601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:43.263603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:43.264127 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:43.264349 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:43.264709 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 09:42:43.264994 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":140,"threads_started":1}
I20260812 09:42:43.266050 17838 leader_election.cc:302] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60e5b015702e475c8f224342b163ad79; no voters:
I20260812 09:42:43.266392 17838 leader_election.cc:288] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:42:43.266502 17844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:42:43.266721 17844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 1 LEADER]: Becoming Leader. State: Replica: 60e5b015702e475c8f224342b163ad79, State: Running, Role: LEADER
I20260812 09:42:43.267074 17844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } }
I20260812 09:42:43.267688 17838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:43.268767 17845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60e5b015702e475c8f224342b163ad79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } } }
I20260812 09:42:43.268905 17845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:43.268965 17846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60e5b015702e475c8f224342b163ad79. Latest consensus state: current_term: 1 leader_uuid: "60e5b015702e475c8f224342b163ad79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } } }
I20260812 09:42:43.269058 17846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:43.269311 17854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:42:43.272048 17854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:42:43.272567 17838 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 09:42:43.285248 17854 catalog_manager.cc:1347] Generated new cluster ID: 8234cbe7734e46579a41c0eff4bdab82
I20260812 09:42:43.285334 17854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:42:43.305917 17854 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:42:43.307354 17854 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:42:43.316079 17854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Generated new TSK 0
I20260812 09:42:43.317018 17854 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:42:44.007788 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:35607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35309
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:44.217851 17888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:44.218247 17888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:44.218329 17888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:44.224819 17888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:44.224915 17888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:44.225032 17888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:42:44.233608 17888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.147.20:35607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:44.235183 17888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:44.236681 17888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:44.250874 17895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:44.251889 17896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:44.252091 17898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:44.253598 17888 server_base.cc:1034] running on GCE node
I20260812 09:42:44.254150 17888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:44.255090 17888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:44.256281 17888 hybrid_clock.cc:648] HybridClock initialized: now 1786527764256202 us; error 94 us; skew 500 ppm
Aug 12 09:42:44 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527764, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:44.260653 17888 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:42:44.261693 17888 webserver.cc:460] Webserver started at http://127.3.147.1:44411/ using document root <none> and password file <none>
I20260812 09:42:44.262391 17888 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:44.262472 17888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:44.262689 17888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 09:42:44 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527764, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:44.281212 17888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "6afc5512db52469ea667631d7c2bd1ee"
format_stamp: "Formatted at 2026-08-12 09:42:44 on dist-test-slave-cbbz"
server_key: "a470e722ae418ee5134885350e1fb186"
server_key_iv: "fed320c64f9aa7626caf5cdc8563fc0e"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:44.282095 17888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "6afc5512db52469ea667631d7c2bd1ee"
format_stamp: "Formatted at 2026-08-12 09:42:44 on dist-test-slave-cbbz"
server_key: "a470e722ae418ee5134885350e1fb186"
server_key_iv: "fed320c64f9aa7626caf5cdc8563fc0e"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:44.293951 17888 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.006s sys 0.000s
I20260812 09:42:44.320039 17905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:44.321892 17888 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.002s
I20260812 09:42:44.322041 17888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "6afc5512db52469ea667631d7c2bd1ee"
format_stamp: "Formatted at 2026-08-12 09:42:44 on dist-test-slave-cbbz"
server_key: "a470e722ae418ee5134885350e1fb186"
server_key_iv: "fed320c64f9aa7626caf5cdc8563fc0e"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:44.322186 17888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:44.352531 17888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:44.356468 17888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:44.356683 17888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:44.357512 17888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:44.358634 17888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:44.358742 17888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:44.358882 17888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:44.358971 17888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:44.376039 17888 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:41427
I20260812 09:42:44.377261 17888 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:42:44.381600 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 17888
I20260812 09:42:44.381732 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:42:44.382099 3660 external_mini_cluster.cc:1428] Setting key 8e5acd08846ba4cf3962af1f24359bac
I20260812 09:42:44.378827 18036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:41427 every 8 connection(s)
Aug 12 09:42:44 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527764, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:42:44.401117 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:44.381522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35309 (local address 127.3.147.1:46735)
0812 09:42:44.384310 (+ 2788us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:44.384366 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:44.385646 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:44.386558 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:44.386567 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:44.387001 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:44.387795 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:44.387819 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:44.389281 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:44.389285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:44.390952 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:44.390965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:44.391719 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:44.391723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:44.391998 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:44.392786 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:44.392807 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:44.394802 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:44.397350 (+ 2548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:44.397358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:44.397376 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:44.397923 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:44.398273 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:44.398277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:44.398279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:44.398457 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:44.399062 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:44.399069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:44.399437 (+ 368us) client_negotiation.cc:769] Sending connection context
0812 09:42:44.399635 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 09:42:44.399982 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2530,"thread_start_us":235,"threads_started":1}
I20260812 09:42:44.402655 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:44.382794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:46735 (local address 127.3.147.62:35309)
0812 09:42:44.383089 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:44.383095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:44.385948 (+ 2853us) server_negotiation.cc:407] Connection header received
0812 09:42:44.385999 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:44.386002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:44.386053 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:44.386152 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:44.388019 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:44.389060 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:44.391130 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:44.391522 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:44.394991 (+ 3469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:44.395165 (+ 174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:44.395169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:44.395172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:44.395193 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:44.397203 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:44.398056 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:44.398059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:44.398061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:44.398168 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:44.398615 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:44.398619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:44.398620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:44.398932 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:44.402214 (+ 3282us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:44.402298 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 09:42:44.402433 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":84,"threads_started":1}
I20260812 09:42:44.403919 18037 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35309
I20260812 09:42:44.404201 18037 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:42:44.404772 18037 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:44.406759 17799 ts_manager.cc:194] Registered new tserver with Master: 6afc5512db52469ea667631d7c2bd1ee (127.3.147.1:41427)
I20260812 09:42:44.408380 17799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:46735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:42:44.435207 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:35607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35309
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:42:44 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786527764, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 09:42:44.622542 18043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:44.622797 18043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:44.622841 18043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:44.627023 18043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:44.627077 18043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:44.627136 18043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:42:44.632871 18043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.147.20:35607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:44.633915 18043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:44.634974 18043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:44.648972 18051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:44.649046 18053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:44.649513 18043 server_base.cc:1034] running on GCE node
W20260812 09:42:44.650797 18050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:44.651361 18043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:44.652179 18043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:44.653383 18043 hybrid_clock.cc:648] HybridClock initialized: now 1786527764653340 us; error 60 us; skew 500 ppm
Aug 12 09:42:44 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527764, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:44.657317 18043 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:42:44.658398 18043 webserver.cc:460] Webserver started at http://127.3.147.2:43795/ using document root <none> and password file <none>
I20260812 09:42:44.659128 18043 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:44.659238 18043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:44.659492 18043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 09:42:44 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527764, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:44.675376 18043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "90ddea4924fd41e5884dbc762c94a696"
format_stamp: "Formatted at 2026-08-12 09:42:44 on dist-test-slave-cbbz"
server_key: "f82dbca3c73d6d8559093700ac017b73"
server_key_iv: "0d667a3815ed90c413273bfcfc08f525"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:44.676488 18043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "90ddea4924fd41e5884dbc762c94a696"
format_stamp: "Formatted at 2026-08-12 09:42:44 on dist-test-slave-cbbz"
server_key: "f82dbca3c73d6d8559093700ac017b73"
server_key_iv: "0d667a3815ed90c413273bfcfc08f525"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:44.682102 18043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 09:42:44.696648 18061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:44.698292 18043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:42:44.698421 18043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "90ddea4924fd41e5884dbc762c94a696"
format_stamp: "Formatted at 2026-08-12 09:42:44 on dist-test-slave-cbbz"
server_key: "f82dbca3c73d6d8559093700ac017b73"
server_key_iv: "0d667a3815ed90c413273bfcfc08f525"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:44.698544 18043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:44.724702 18043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:44.728839 18043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:44.729121 18043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:44.729935 18043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:44.731207 18043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:44.731333 18043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:44.731436 18043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:44.731504 18043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:44.750854 18043 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:42485
I20260812 09:42:44.753545 18188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:42485 every 8 connection(s)
I20260812 09:42:44.754050 18043 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:42:44.755224 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 18043
I20260812 09:42:44.755355 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:42:44.755673 3660 external_mini_cluster.cc:1428] Setting key d2079689ed1747af73231d2a862b5159
Aug 12 09:42:44 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527764, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:44.779309 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:44.761412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:45413 (local address 127.3.147.62:35309)
0812 09:42:44.761566 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:44.761572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:44.763480 (+ 1908us) server_negotiation.cc:407] Connection header received
0812 09:42:44.766607 (+ 3127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:44.766611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:44.766670 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:44.766747 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:44.768528 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:44.769374 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:44.771113 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:44.771487 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:44.771569 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:44.774937 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:44.774941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:44.774944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:44.774973 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:44.776714 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:44.777374 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:44.777377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:44.777379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:44.777512 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:44.777957 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:44.777960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:44.777962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:44.778183 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:44.778266 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:44.778996 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 09:42:44.779131 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 09:42:44.781178 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:44.761968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35309 (local address 127.3.147.2:45413)
0812 09:42:44.763319 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:44.763379 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:44.764715 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:44.767168 (+ 2453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:44.767177 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:44.767573 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:44.768349 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:44.768372 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:44.769545 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:44.769548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:44.770956 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:44.770967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:44.771819 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:44.771823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:44.772030 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:44.772813 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:44.772834 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:44.774744 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:44.776847 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:44.776853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:44.776870 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:44.777256 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:44.777626 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:44.777629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:44.777631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:44.777844 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:44.778333 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:44.778340 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:44.778713 (+ 373us) client_negotiation.cc:769] Sending connection context
0812 09:42:44.779744 (+ 1031us) client_negotiation.cc:240] Negotiation successful
0812 09:42:44.780077 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"thread_start_us":202,"threads_started":1}
I20260812 09:42:44.782804 18189 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35309
I20260812 09:42:44.783054 18189 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:44.783619 18189 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:44.784886 17799 ts_manager.cc:194] Registered new tserver with Master: 90ddea4924fd41e5884dbc762c94a696 (127.3.147.2:42485)
I20260812 09:42:44.785753 17799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:45413
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:42:44.824707 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:35607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35309
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:45.004896 18195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:45.005228 18195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:45.005348 18195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:45.009516 18195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:45.009598 18195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:45.009713 18195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:42:45.015203 18195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.147.20:35607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:45.016414 18195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:45.017772 18195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:45.030184 18203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:45.030665 18202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:45.031390 18195 server_base.cc:1034] running on GCE node
W20260812 09:42:45.031790 18205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:45.032347 18195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:45.033092 18195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:45.040760 18195 hybrid_clock.cc:648] HybridClock initialized: now 1786527765040653 us; error 100 us; skew 500 ppm
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:45.045523 18195 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:42:45.046850 18195 webserver.cc:460] Webserver started at http://127.3.147.3:43125/ using document root <none> and password file <none>
I20260812 09:42:45.047640 18195 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:45.047788 18195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:45.048069 18195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527764, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:45.081275 18195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "c03ccf6afab442a284cef73a1fec15a0"
format_stamp: "Formatted at 2026-08-12 09:42:45 on dist-test-slave-cbbz"
server_key: "e903ef213d75f749def3f37abf1d6801"
server_key_iv: "fed96d169025c562938620786ed389c6"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:45.082086 18195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "c03ccf6afab442a284cef73a1fec15a0"
format_stamp: "Formatted at 2026-08-12 09:42:45 on dist-test-slave-cbbz"
server_key: "e903ef213d75f749def3f37abf1d6801"
server_key_iv: "fed96d169025c562938620786ed389c6"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:45.100438 18195 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.006s sys 0.001s
I20260812 09:42:45.132716 18212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.134431 18195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:42:45.134563 18195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "c03ccf6afab442a284cef73a1fec15a0"
format_stamp: "Formatted at 2026-08-12 09:42:45 on dist-test-slave-cbbz"
server_key: "e903ef213d75f749def3f37abf1d6801"
server_key_iv: "fed96d169025c562938620786ed389c6"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:45.134711 18195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:45.160113 18195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:45.164057 18195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:45.164362 18195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:45.165210 18195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:45.166393 18195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:45.166500 18195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.166615 18195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:45.166689 18195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.186308 18195 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42403
I20260812 09:42:45.187695 18195 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:42:45.189996 18338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42403 every 8 connection(s)
I20260812 09:42:45.197803 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 18195
I20260812 09:42:45.197928 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:42:45.198241 3660 external_mini_cluster.cc:1428] Setting key c329c50b175fdd63f4d9d9509537422b
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:45.213850 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:45.193444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:45731 (local address 127.3.147.62:35309)
0812 09:42:45.193585 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:45.193590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:45.194688 (+ 1098us) server_negotiation.cc:407] Connection header received
0812 09:42:45.196093 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:45.196099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:45.196187 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:45.196244 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:45.198134 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:45.199012 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:45.203135 (+ 4123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:45.203504 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:45.203611 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:45.207246 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:45.207251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:45.207254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:45.207286 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:45.209324 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:45.210027 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:45.210031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:45.210033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:45.210146 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:45.210629 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:45.210633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:45.210635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:45.210876 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:45.210957 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:45.213517 (+ 2560us) server_negotiation.cc:299] Negotiation successful
0812 09:42:45.213671 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:42:45.215852 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:45.192299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35309 (local address 127.3.147.3:45731)
0812 09:42:45.194491 (+ 2192us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:45.194547 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:45.195827 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:45.196663 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:45.196672 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:45.197094 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:45.197830 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:45.197850 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:45.201430 (+ 3580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:45.201433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:45.202964 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:45.202978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:45.203837 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:45.203842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:45.204058 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:45.204793 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:45.204816 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:45.207029 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:45.209491 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:45.209497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:45.209514 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:45.209890 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:45.210290 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:45.210294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:45.210296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:45.210484 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:45.211045 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:45.211052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:45.213183 (+ 2131us) client_negotiation.cc:769] Sending connection context
0812 09:42:45.214327 (+ 1144us) client_negotiation.cc:240] Negotiation successful
0812 09:42:45.214686 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"thread_start_us":173,"threads_started":1}
I20260812 09:42:45.217243 18339 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35309
I20260812 09:42:45.217520 18339 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:45.218151 18339 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:45.219384 17799 ts_manager.cc:194] Registered new tserver with Master: c03ccf6afab442a284cef73a1fec15a0 (127.3.147.3:42403)
I20260812 09:42:45.220235 17799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:45731
I20260812 09:42:45.225414 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:42:45.226358 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 17888
I20260812 09:42:45.244174 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 18043
I20260812 09:42:45.254940 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 18195
I20260812 09:42:45.265156 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 17761
I20260812 09:42:45.276847 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35309
--webserver_interface=127.3.147.62
--webserver_port=40173
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:35309
--ranger_config_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:35607/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.147.20:35607/kms with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:45.388031 18343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:45.388302 18343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:45.388370 18343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:45.392364 18343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:45.392413 18343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:45.392432 18343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:45.392447 18343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:45.392462 18343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:45.397728 18343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.147.20:35607/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:35309
--ranger_config_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:45.398723 18343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:45.399734 18343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:45.407886 18350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:45.407960 18349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:45.408030 18352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:45.408205 18343 server_base.cc:1034] running on GCE node
I20260812 09:42:45.408860 18343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:45.409885 18343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:45.411075 18343 hybrid_clock.cc:648] HybridClock initialized: now 1786527765411052 us; error 45 us; skew 500 ppm
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:45.413846 18343 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:42:45.414686 18343 webserver.cc:460] Webserver started at http://127.3.147.62:40173/ using document root <none> and password file <none>
I20260812 09:42:45.415222 18343 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:45.415320 18343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:45.454685 18343 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:42:45.457121 18361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.458532 18343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:42:45.458685 18343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "60e5b015702e475c8f224342b163ad79"
format_stamp: "Formatted at 2026-08-12 09:42:43 on dist-test-slave-cbbz"
server_key: "1d9ec07a0b2286d5aae6b83963ef0ca8"
server_key_iv: "24d666f480e5598204791ddb08c12e51"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:45.459256 18343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:45.478510 18343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:45.481578 18343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:45.481791 18343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:45.489240 18343 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:35309
I20260812 09:42:45.489290 18422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:35309 every 8 connection(s)
I20260812 09:42:45.490336 18343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:42:45.494117 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 18343
I20260812 09:42:45.494560 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:35607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:41427
--local_ip_for_outbound_sockets=127.3.147.1
--tserver_master_addrs=127.3.147.62:35309
--webserver_port=44411
--webserver_interface=127.3.147.1
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.147.20:35607/kms with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:42:45.494637 18423 sys_catalog.cc:263] Verifying existing consensus state
I20260812 09:42:45.496315 18423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Bootstrap starting.
I20260812 09:42:45.513293 18423 log.cc:824] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:45.519906 18423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:42:45.520371 18423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Bootstrap complete.
I20260812 09:42:45.524700 18423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } }
I20260812 09:42:45.525108 18423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60e5b015702e475c8f224342b163ad79, State: Initialized, Role: FOLLOWER
I20260812 09:42:45.525629 18423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } }
I20260812 09:42:45.525779 18423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:42:45.525869 18423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:42:45.525982 18423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 1 FOLLOWER]: Advancing to term 2
I20260812 09:42:45.527840 18423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } }
I20260812 09:42:45.528210 18423 leader_election.cc:302] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60e5b015702e475c8f224342b163ad79; no voters:
I20260812 09:42:45.528565 18423 leader_election.cc:288] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 09:42:45.528709 18427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 09:42:45.528950 18427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [term 2 LEADER]: Becoming Leader. State: Replica: 60e5b015702e475c8f224342b163ad79, State: Running, Role: LEADER
I20260812 09:42:45.529359 18427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } }
I20260812 09:42:45.529794 18423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:45.530812 18429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60e5b015702e475c8f224342b163ad79. Latest consensus state: current_term: 2 leader_uuid: "60e5b015702e475c8f224342b163ad79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } } }
I20260812 09:42:45.530954 18429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:45.531226 18428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "60e5b015702e475c8f224342b163ad79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e5b015702e475c8f224342b163ad79" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35309 } } }
I20260812 09:42:45.531318 18428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79 [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:45.531392 18438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:42:45.534644 18438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:42:45.535082 18423 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 09:42:45.535401 18438 catalog_manager.cc:1259] Loaded cluster ID: 8234cbe7734e46579a41c0eff4bdab82
I20260812 09:42:45.535488 18438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:42:45.541481 18438 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:42:45.542330 18438 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Loaded TSK: 0
I20260812 09:42:45.543579 18438 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 09:42:45.618328 18425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:45.618606 18425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:45.618692 18425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:45.623148 18425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:45.623236 18425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:45.623353 18425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:42:45.629447 18425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.147.20:35607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:41427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=44411
--tserver_master_addrs=127.3.147.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:45.630682 18425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:45.631827 18425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:45.643966 18468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:45.644006 18466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:45.644001 18465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:45.645711 18425 server_base.cc:1034] running on GCE node
I20260812 09:42:45.646241 18425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:45.646987 18425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:45.648226 18425 hybrid_clock.cc:648] HybridClock initialized: now 1786527765647999 us; error 232 us; skew 500 ppm
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:45.651489 18425 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:42:45.652680 18425 webserver.cc:460] Webserver started at http://127.3.147.1:44411/ using document root <none> and password file <none>
I20260812 09:42:45.653311 18425 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:45.653432 18425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:45.684338 18425 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:42:45.687959 18479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.689437 18425 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 09:42:45.689618 18425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "6afc5512db52469ea667631d7c2bd1ee"
format_stamp: "Formatted at 2026-08-12 09:42:44 on dist-test-slave-cbbz"
server_key: "a470e722ae418ee5134885350e1fb186"
server_key_iv: "fed320c64f9aa7626caf5cdc8563fc0e"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:45.690194 18425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:45.705973 18425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:45.709785 18425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:45.710031 18425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:45.710856 18425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:45.712042 18425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:45.712126 18425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.712209 18425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:45.712242 18425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.726960 18425 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:41427
I20260812 09:42:45.727015 18613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:41427 every 8 connection(s)
I20260812 09:42:45.728416 18425 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:42:45.737165 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 18425
I20260812 09:42:45.737754 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:35607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:42485
--local_ip_for_outbound_sockets=127.3.147.2
--tserver_master_addrs=127.3.147.62:35309
--webserver_port=43795
--webserver_interface=127.3.147.2
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.147.20:35607/kms with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:45.755983 18617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:45.731089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35309 (local address 127.3.147.1:37429)
0812 09:42:45.735629 (+ 4540us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:45.735687 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:45.736778 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:45.738962 (+ 2184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:45.738972 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:45.739369 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:45.739984 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:45.740004 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:45.741890 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:45.741893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:45.743350 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:45.743362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:45.744697 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:45.744701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:45.744947 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:45.746775 (+ 1828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:45.746797 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:45.748790 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:45.751743 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:45.751750 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:45.751766 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:45.752196 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:45.752670 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:45.752674 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:45.752677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:45.752864 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:45.753789 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:45.753800 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:45.754195 (+ 395us) client_negotiation.cc:769] Sending connection context
0812 09:42:45.754429 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 09:42:45.754814 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4328,"thread_start_us":151,"threads_started":1}
I20260812 09:42:45.758330 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:45.731094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:37429 (local address 127.3.147.62:35309)
0812 09:42:45.735337 (+ 4243us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:45.735361 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:45.735853 (+ 492us) server_negotiation.cc:407] Connection header received
0812 09:42:45.737482 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:45.737498 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:45.738133 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:45.738483 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:45.740212 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:45.741709 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:45.743514 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:45.744275 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:45.744440 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:45.749107 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:45.749115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:45.749134 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:45.749173 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:45.751575 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:45.752364 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:45.752371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:45.752379 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:45.752538 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:45.752991 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:45.752995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:45.752997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:45.753563 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:45.753721 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:45.756914 (+ 3193us) server_negotiation.cc:299] Negotiation successful
0812 09:42:45.757263 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4056,"thread_start_us":129,"threads_started":1}
I20260812 09:42:45.762259 18614 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35309
I20260812 09:42:45.762526 18614 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:45.763233 18614 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
W20260812 09:42:45.766106 18614 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:35309 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 09:42:45.766249 18614 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:45.766472 18614 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:45.767683 18614 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:45.767896 18614 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
W20260812 09:42:45.768849 18614 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 09:42:45.892302 18619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:45.892601 18619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:45.892669 18619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:45.897465 18619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:45.897547 18619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:45.897639 18619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:42:45.904192 18619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.147.20:35607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:42485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=43795
--tserver_master_addrs=127.3.147.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:45.905535 18619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:45.906889 18619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:45.919107 18625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:45.919247 18627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:45.919430 18624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:45.919795 18619 server_base.cc:1034] running on GCE node
I20260812 09:42:45.920253 18619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:45.921070 18619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:45.922258 18619 hybrid_clock.cc:648] HybridClock initialized: now 1786527765922189 us; error 87 us; skew 500 ppm
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:45.925213 18619 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:42:45.926219 18619 webserver.cc:460] Webserver started at http://127.3.147.2:43795/ using document root <none> and password file <none>
I20260812 09:42:45.926819 18619 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:45.926919 18619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 09:42:45 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:45.948886 18619 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 09:42:45.952325 18635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.954231 18619 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 09:42:45.954340 18619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "90ddea4924fd41e5884dbc762c94a696"
format_stamp: "Formatted at 2026-08-12 09:42:44 on dist-test-slave-cbbz"
server_key: "f82dbca3c73d6d8559093700ac017b73"
server_key_iv: "0d667a3815ed90c413273bfcfc08f525"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:45.954924 18619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:45.985615 18619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:45.989279 18619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:45.989490 18619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:45.990198 18619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:45.991252 18619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:45.991313 18619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:45.991364 18619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:45.991393 18619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:46.002713 18619 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:42485
I20260812 09:42:46.002889 18767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:42485 every 8 connection(s)
I20260812 09:42:46.003784 18619 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:42:46.011457 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 18619
I20260812 09:42:46.012482 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.147.20:35607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:42403
--local_ip_for_outbound_sockets=127.3.147.3
--tserver_master_addrs=127.3.147.62:35309
--webserver_port=43125
--webserver_interface=127.3.147.3
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.147.20:35607/kms with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:42:46 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527765, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:46.021143 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:46.005745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:34613 (local address 127.3.147.62:35309)
0812 09:42:46.005879 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:46.005884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:46.006825 (+ 941us) server_negotiation.cc:407] Connection header received
0812 09:42:46.007826 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:46.007830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:46.007897 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:46.007973 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:46.009079 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.009616 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.010733 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.010974 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.011087 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:46.016641 (+ 5554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:46.016645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:46.016648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:46.016680 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:46.018582 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:46.019261 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:46.019265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:46.019267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:46.019356 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:46.019780 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:46.019783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:46.019785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:46.020009 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:46.020084 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:46.020826 (+ 742us) server_negotiation.cc:299] Negotiation successful
0812 09:42:46.020970 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:42:46.022975 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:46.006188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35309 (local address 127.3.147.2:34613)
0812 09:42:46.006661 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:46.006722 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:46.007606 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:46.008184 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:46.008193 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:46.008462 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:46.008927 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:46.008940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.009755 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.009763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:46.010594 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:46.010603 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.011138 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.011142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:46.011351 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:46.012018 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:46.012039 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:46.016414 (+ 4375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:46.018734 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:46.018739 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:46.018754 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:46.019117 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:46.019478 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:46.019481 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:46.019483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:46.019656 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:46.020222 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:46.020228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:46.020541 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 09:42:46.021584 (+ 1043us) client_negotiation.cc:240] Negotiation successful
0812 09:42:46.021953 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":133,"threads_started":1}
I20260812 09:42:46.024587 18768 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35309
I20260812 09:42:46.024858 18768 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:46.025439 18768 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
W20260812 09:42:46.027061 18768 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:35309 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 09:42:46.027196 18768 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:46.027388 18768 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:46.028228 18768 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:46.028402 18768 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
W20260812 09:42:46.029158 18768 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 09:42:46.177589 18772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:46.177994 18772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:46.178105 18772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:46.185616 18772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:46.185683 18772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:46.185743 18772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:42:46.191872 18772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.147.20:35607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:42403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=43125
--tserver_master_addrs=127.3.147.62:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:46.193044 18772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:46.194128 18772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:46.204238 18780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:46.204329 18779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:46.204445 18772 server_base.cc:1034] running on GCE node
W20260812 09:42:46.204339 18782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:46.205098 18772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:46.205833 18772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:46.207003 18772 hybrid_clock.cc:648] HybridClock initialized: now 1786527766206976 us; error 45 us; skew 500 ppm
Aug 12 09:42:46 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527766, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:46.209873 18772 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:42:46.210867 18772 webserver.cc:460] Webserver started at http://127.3.147.3:43125/ using document root <none> and password file <none>
I20260812 09:42:46.211400 18772 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:46.211462 18772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 09:42:46 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527766, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:46.243587 18772 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260812 09:42:46.246528 18790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:46.247952 18772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:42:46.248108 18772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "c03ccf6afab442a284cef73a1fec15a0"
format_stamp: "Formatted at 2026-08-12 09:42:45 on dist-test-slave-cbbz"
server_key: "e903ef213d75f749def3f37abf1d6801"
server_key_iv: "fed96d169025c562938620786ed389c6"
server_key_version: "kuduclusterkey@0"
I20260812 09:42:46.248791 18772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:46.274629 18772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:46.278467 18772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:46.278692 18772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:46.279461 18772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:46.280598 18772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:46.280669 18772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:46.280736 18772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:46.280756 18772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:46.291882 18772 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42403
I20260812 09:42:46.292224 18912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42403 every 8 connection(s)
I20260812 09:42:46.293279 18772 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:42:46.300712 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 18772
I20260812 09:42:46.317519 18917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:46.301894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49788 (local address 127.3.147.62:35309)
0812 09:42:46.304707 (+ 2813us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:46.304713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:46.304729 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:42:46.304779 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:46.304782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:46.304832 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:46.304922 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:46.305914 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.306757 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.309514 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.309859 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.309983 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:46.311790 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:46.311794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:46.311796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:46.311823 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:46.313691 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:46.314189 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:46.314193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:46.314203 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:46.314310 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:46.314632 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:46.314636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:46.314638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:46.314878 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:46.315112 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:46.317141 (+ 2029us) server_negotiation.cc:299] Negotiation successful
0812 09:42:46.317311 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2736,"thread_start_us":89,"threads_started":1}
Aug 12 09:42:46 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527766, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:46.322813 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:46.296249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:58051 (local address 127.3.147.62:35309)
0812 09:42:46.296408 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:46.296413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:46.308661 (+ 12248us) server_negotiation.cc:407] Connection header received
0812 09:42:46.310450 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:46.310454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:46.310511 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:46.310581 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:46.312449 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.313320 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.314988 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.315290 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.315398 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:46.318429 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:46.318434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:46.318438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:46.318464 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:46.320280 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:46.320945 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:46.320950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:46.320953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:46.321043 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:46.321482 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:46.321485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:46.321486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:46.321709 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:46.321830 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:46.322500 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 09:42:46.322641 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 09:42:46.323835 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:46.296789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:35309 (local address 127.3.147.3:58051)
0812 09:42:46.308454 (+ 11665us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:46.308542 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:46.309781 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:46.311080 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:46.311090 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:46.311466 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:46.312259 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:46.312289 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.313526 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.313529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:46.314866 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:46.314875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:46.315477 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:46.315482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:46.315695 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:46.316316 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:46.316338 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:46.318146 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:46.320423 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:46.320429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:46.320444 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:46.320812 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:46.321166 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:46.321169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:46.321171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:46.321356 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:46.321850 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:46.321857 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:46.322186 (+ 329us) client_negotiation.cc:769] Sending connection context
0812 09:42:46.322406 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 09:42:46.322755 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7052,"thread_start_us":164,"threads_started":1}
I20260812 09:42:46.325321 18913 heartbeater.cc:347] Connected to a master server at 127.3.147.62:35309
I20260812 09:42:46.325613 18913 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:46.326233 18913 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
W20260812 09:42:46.327929 18913 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:35309 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 09:42:46.328066 18913 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:46.328274 18913 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:46.329093 18913 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:46.329268 18913 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
W20260812 09:42:46.330081 18913 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 09:42:46 dist-test-slave-cbbz krb5kdc[17273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786527766, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:46.769120 18614 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:46.769501 18614 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:46.770741 18379 ts_manager.cc:194] Registered new tserver with Master: 6afc5512db52469ea667631d7c2bd1ee (127.3.147.1:41427)
I20260812 09:42:46.772441 18379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:37429
I20260812 09:42:47.029395 18768 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:47.029726 18768 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:47.030519 18379 ts_manager.cc:194] Registered new tserver with Master: 90ddea4924fd41e5884dbc762c94a696 (127.3.147.2:42485)
I20260812 09:42:47.031364 18379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:34613
Aug 12 09:42:47 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527766, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
I20260812 09:42:47.330313 18913 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:47.330641 18913 heartbeater.cc:511] Master 127.3.147.62:35309 requested a full tablet report, sending...
I20260812 09:42:47.331449 18379 ts_manager.cc:194] Registered new tserver with Master: c03ccf6afab442a284cef73a1fec15a0 (127.3.147.3:42403)
I20260812 09:42:47.332356 18379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:58051
I20260812 09:42:47.337767 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:42:47.350062 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.339599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.3.147.62:35309)
0812 09:42:47.339910 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:47.339915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:47.340239 (+ 324us) server_negotiation.cc:407] Connection header received
0812 09:42:47.340401 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:47.340405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:47.340456 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:47.340539 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:47.341445 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.342271 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.343500 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.344059 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.344291 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.345994 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:47.345999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:47.346001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:47.346028 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:47.347946 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:47.348461 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:47.348465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:47.348467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:47.348550 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:47.348822 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:47.348825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:47.348827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:47.349020 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:47.349142 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:47.349778 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 09:42:47.349898 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":137,"threads_started":1}
I20260812 09:42:47.354532 18379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:42:47.359122 18379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:42:47.380800 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.372265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42403 (local address 127.0.0.1:50326)
0812 09:42:47.374848 (+ 2583us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:47.374862 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:47.374958 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:47.375828 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:47.375831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:47.375848 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:47.375951 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.375958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.377457 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.377461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.378859 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.378870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.380218 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.380222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.380338 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.380384 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 09:42:47.380501 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 09:42:47.380633 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2518,"thread_start_us":102,"threads_started":1}
I20260812 09:42:47.381791 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.372680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.3.147.3:42403)
0812 09:42:47.375218 (+ 2538us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:47.375225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:47.375250 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:42:47.375326 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:47.375332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:47.375513 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:47.375662 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:47.376104 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.377317 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.379004 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.379815 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.379924 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.380112 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:47.381487 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0812 09:42:47.381609 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2426,"thread_start_us":88,"threads_started":1}
I20260812 09:42:47.381980 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.371537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45318 (local address 127.3.147.1:41427)
0812 09:42:47.373406 (+ 1869us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:47.373413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:47.374110 (+ 697us) server_negotiation.cc:407] Connection header received
0812 09:42:47.374195 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:47.374201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:47.374402 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:47.374561 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:47.375045 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.377952 (+ 2907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.379385 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.380110 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.380253 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.380410 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:47.381689 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0812 09:42:47.381787 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1755,"thread_start_us":114,"threads_started":1}
I20260812 09:42:47.382157 18950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.371391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:41427 (local address 127.0.0.1:45318)
0812 09:42:47.373761 (+ 2370us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:47.373815 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:47.373956 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:47.374576 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:47.374582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:47.374608 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:47.374804 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.374813 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.378100 (+ 3287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.378104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.379239 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.379250 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.380472 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.380476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.380567 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.380592 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 09:42:47.380655 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 09:42:47.380725 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2166,"thread_start_us":171,"threads_started":1}
I20260812 09:42:47.385560 18828 tablet_service.cc:1467] Processing CreateTablet for tablet 7176a90625e84eb8b476e7e8cbef0e9b (DEFAULT_TABLE table=test-table [id=dcddb7637e644f6d848b37886ac1cf44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:47.386114 18536 tablet_service.cc:1467] Processing CreateTablet for tablet 7176a90625e84eb8b476e7e8cbef0e9b (DEFAULT_TABLE table=test-table [id=dcddb7637e644f6d848b37886ac1cf44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:47.386745 18828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7176a90625e84eb8b476e7e8cbef0e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:47.387042 18536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7176a90625e84eb8b476e7e8cbef0e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:47.388763 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.372786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.3.147.2:42485)
0812 09:42:47.373226 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:47.373233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:47.380533 (+ 7300us) server_negotiation.cc:407] Connection header received
0812 09:42:47.380625 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:47.380634 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:47.380860 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:47.381007 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:47.381415 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.382550 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.383998 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.384950 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.385071 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.385230 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:47.388534 (+ 3304us) server_negotiation.cc:299] Negotiation successful
0812 09:42:47.388630 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":169,"threads_started":1}
I20260812 09:42:47.389151 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.372320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:42485 (local address 127.0.0.1:42198)
0812 09:42:47.380306 (+ 7986us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:47.380321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:47.380417 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:47.381167 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:47.381170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:47.381190 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:47.381273 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.381278 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.382702 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.382706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.383815 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.383826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.388250 (+ 4424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.388254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.388338 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.388379 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 09:42:47.388957 (+ 578us) client_negotiation.cc:240] Negotiation successful
0812 09:42:47.389034 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7919,"thread_start_us":164,"threads_started":1}
I20260812 09:42:47.391927 18687 tablet_service.cc:1467] Processing CreateTablet for tablet 7176a90625e84eb8b476e7e8cbef0e9b (DEFAULT_TABLE table=test-table [id=dcddb7637e644f6d848b37886ac1cf44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:47.392940 18687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7176a90625e84eb8b476e7e8cbef0e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:47.403115 18955 tablet_bootstrap.cc:492] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Bootstrap starting.
I20260812 09:42:47.404507 18953 tablet_bootstrap.cc:492] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee: Bootstrap starting.
I20260812 09:42:47.406966 18953 tablet_bootstrap.cc:654] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:47.407716 18954 tablet_bootstrap.cc:492] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Bootstrap starting.
I20260812 09:42:47.407967 18953 log.cc:824] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:47.408804 18955 tablet_bootstrap.cc:654] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:47.409976 18955 log.cc:824] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:47.410327 18954 tablet_bootstrap.cc:654] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:47.411207 18954 log.cc:824] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:47.413131 18955 tablet_bootstrap.cc:492] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: No bootstrap required, opened a new log
I20260812 09:42:47.413345 18955 ts_tablet_manager.cc:1397] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 09:42:47.413462 18954 tablet_bootstrap.cc:492] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: No bootstrap required, opened a new log
I20260812 09:42:47.413690 18954 ts_tablet_manager.cc:1397] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:42:47.418913 18955 raft_consensus.cc:348] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.419211 18955 raft_consensus.cc:374] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:47.419282 18955 raft_consensus.cc:729] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ddea4924fd41e5884dbc762c94a696, State: Initialized, Role: FOLLOWER
I20260812 09:42:47.419768 18955 consensus_queue.cc:260] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.420222 18954 raft_consensus.cc:348] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.420536 18954 raft_consensus.cc:374] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:47.420686 18954 raft_consensus.cc:729] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03ccf6afab442a284cef73a1fec15a0, State: Initialized, Role: FOLLOWER
I20260812 09:42:47.420892 18955 ts_tablet_manager.cc:1428] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 09:42:47.421278 18768 heartbeater.cc:503] Master 127.3.147.62:35309 was elected leader, sending a full tablet report...
I20260812 09:42:47.421272 18954 consensus_queue.cc:260] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.422164 18953 tablet_bootstrap.cc:492] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee: No bootstrap required, opened a new log
I20260812 09:42:47.422417 18953 ts_tablet_manager.cc:1397] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260812 09:42:47.422667 18954 ts_tablet_manager.cc:1428] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260812 09:42:47.423058 18913 heartbeater.cc:503] Master 127.3.147.62:35309 was elected leader, sending a full tablet report...
I20260812 09:42:47.427385 18953 raft_consensus.cc:348] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.427639 18953 raft_consensus.cc:374] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:47.427742 18953 raft_consensus.cc:729] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afc5512db52469ea667631d7c2bd1ee, State: Initialized, Role: FOLLOWER
I20260812 09:42:47.428264 18953 consensus_queue.cc:260] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.429322 18953 ts_tablet_manager.cc:1428] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 09:42:47.429670 18614 heartbeater.cc:503] Master 127.3.147.62:35309 was elected leader, sending a full tablet report...
Aug 12 09:42:47 dist-test-slave-cbbz krb5kdc[17273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527766, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for HTTP/127.3.147.20@KRBTEST.COM
W20260812 09:42:47.486565 18615 tablet.cc:2369] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:42:47.506248 18769 tablet.cc:2369] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:42:47.520733 18963 raft_consensus.cc:482] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:42:47.520977 18963 raft_consensus.cc:504] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.522383 18963 leader_election.cc:288] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c03ccf6afab442a284cef73a1fec15a0 (127.3.147.3:42403), 90ddea4924fd41e5884dbc762c94a696 (127.3.147.2:42485)
I20260812 09:42:47.529049 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.523151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:59127 (local address 127.3.147.2:42485)
0812 09:42:47.523284 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:47.523289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:47.523848 (+ 559us) server_negotiation.cc:407] Connection header received
0812 09:42:47.524070 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:47.524073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:47.524162 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:47.524214 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:47.524878 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.525633 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.527631 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.528254 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.528334 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.528402 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:47.528814 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 09:42:47.528909 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:42:47.529523 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.522646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:42485 (local address 127.3.147.1:59127)
0812 09:42:47.523746 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:47.523764 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:47.523936 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:47.524362 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:47.524365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:47.524383 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:47.524736 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.524743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.525778 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.525781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.527486 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.527497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.528501 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.528505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.528645 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.528672 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 09:42:47.529294 (+ 622us) client_negotiation.cc:240] Negotiation successful
0812 09:42:47.529379 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"thread_start_us":145,"threads_started":1}
I20260812 09:42:47.530246 18711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7176a90625e84eb8b476e7e8cbef0e9b" candidate_uuid: "6afc5512db52469ea667631d7c2bd1ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ddea4924fd41e5884dbc762c94a696" is_pre_election: true
I20260812 09:42:47.530614 18711 raft_consensus.cc:2454] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6afc5512db52469ea667631d7c2bd1ee in term 0.
I20260812 09:42:47.531237 18483 leader_election.cc:302] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6afc5512db52469ea667631d7c2bd1ee, 90ddea4924fd41e5884dbc762c94a696; no voters:
I20260812 09:42:47.532066 18963 raft_consensus.cc:2781] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:42:47.532220 18963 raft_consensus.cc:482] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:42:47.532274 18963 raft_consensus.cc:3037] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:47.533936 18963 raft_consensus.cc:504] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.534340 18963 leader_election.cc:288] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [CANDIDATE]: Term 1 election: Requested vote from peers c03ccf6afab442a284cef73a1fec15a0 (127.3.147.3:42403), 90ddea4924fd41e5884dbc762c94a696 (127.3.147.2:42485)
I20260812 09:42:47.534978 18711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7176a90625e84eb8b476e7e8cbef0e9b" candidate_uuid: "6afc5512db52469ea667631d7c2bd1ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ddea4924fd41e5884dbc762c94a696"
I20260812 09:42:47.535125 18711 raft_consensus.cc:3037] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:47.536785 18711 raft_consensus.cc:2454] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6afc5512db52469ea667631d7c2bd1ee in term 1.
I20260812 09:42:47.537180 18483 leader_election.cc:302] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6afc5512db52469ea667631d7c2bd1ee, 90ddea4924fd41e5884dbc762c94a696; no voters:
I20260812 09:42:47.537356 18963 raft_consensus.cc:2781] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:42:47.537569 18963 raft_consensus.cc:686] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [term 1 LEADER]: Becoming Leader. State: Replica: 6afc5512db52469ea667631d7c2bd1ee, State: Running, Role: LEADER
I20260812 09:42:47.537940 18963 consensus_queue.cc:237] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } }
I20260812 09:42:47.541515 18378 catalog_manager.cc:5495] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 6afc5512db52469ea667631d7c2bd1ee (127.3.147.1). New cstate: current_term: 1 leader_uuid: "6afc5512db52469ea667631d7c2bd1ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afc5512db52469ea667631d7c2bd1ee" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 41427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 } health_report { overall_health: UNKNOWN } } }
I20260812 09:42:47.542546 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.523411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:36979 (local address 127.3.147.3:42403)
0812 09:42:47.523579 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:47.523584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:47.530104 (+ 6520us) server_negotiation.cc:407] Connection header received
0812 09:42:47.530300 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:47.530303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:47.530358 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:47.530405 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:47.530743 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.531792 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.541180 (+ 9388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.541824 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.541911 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.541979 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:47.542325 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 09:42:47.542414 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 09:42:47.542939 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.522769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42403 (local address 127.3.147.1:36979)
0812 09:42:47.530012 (+ 7243us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:47.530028 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:47.530183 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:47.530534 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:47.530537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:47.530552 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:47.530623 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.530628 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.539925 (+ 9297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.539929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.540988 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:47.541011 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.542076 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.542092 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:47.542182 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.542205 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:42:47.542736 (+ 531us) client_negotiation.cc:240] Negotiation successful
0812 09:42:47.542816 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7182,"thread_start_us":109,"threads_started":1}
I20260812 09:42:47.543769 18865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7176a90625e84eb8b476e7e8cbef0e9b" candidate_uuid: "6afc5512db52469ea667631d7c2bd1ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03ccf6afab442a284cef73a1fec15a0" is_pre_election: true
I20260812 09:42:47.544090 18865 raft_consensus.cc:2454] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6afc5512db52469ea667631d7c2bd1ee in term 0.
I20260812 09:42:47.544576 18864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7176a90625e84eb8b476e7e8cbef0e9b" candidate_uuid: "6afc5512db52469ea667631d7c2bd1ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03ccf6afab442a284cef73a1fec15a0"
I20260812 09:42:47.544691 18864 raft_consensus.cc:3037] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:47.546324 18864 raft_consensus.cc:2454] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6afc5512db52469ea667631d7c2bd1ee in term 1.
W20260812 09:42:47.547083 18914 tablet.cc:2369] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:42:47.572827 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.567306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.3.147.1:41427)
0812 09:42:47.567486 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:47.567491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:47.567506 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:42:47.567596 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:47.567600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:47.567693 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:47.567814 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:42:47.568278 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.569086 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.570378 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:47.570721 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:47.570829 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:47.571308 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:42:47.571907 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:42:47.572033 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:47.572536 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 09:42:47.572608 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 09:42:47.581146 18711 raft_consensus.cc:1261] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 1 FOLLOWER]: Refusing update from remote peer 6afc5512db52469ea667631d7c2bd1ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:42:47.582379 18969 consensus_queue.cc:1035] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ddea4924fd41e5884dbc762c94a696" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 42485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:42:47.595321 18864 raft_consensus.cc:1261] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 1 FOLLOWER]: Refusing update from remote peer 6afc5512db52469ea667631d7c2bd1ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:42:47.596163 18969 consensus_queue.cc:1035] T 7176a90625e84eb8b476e7e8cbef0e9b P 6afc5512db52469ea667631d7c2bd1ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "c03ccf6afab442a284cef73a1fec15a0" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:42:47.609493 18972 mvcc.cc:205] Tried to move back new op lower bound from 7317617736002297856 to 7317617735838769152. Current Snapshot: MvccSnapshot[applied={T|T < 7317617736002297856}]
I20260812 09:42:47.612723 18973 mvcc.cc:205] Tried to move back new op lower bound from 7317617736002297856 to 7317617735838769152. Current Snapshot: MvccSnapshot[applied={T|T < 7317617736002297856}]
I20260812 09:42:47.616046 18976 mvcc.cc:205] Tried to move back new op lower bound from 7317617736002297856 to 7317617735838769152. Current Snapshot: MvccSnapshot[applied={T|T < 7317617736002297856}]
I20260812 09:42:47.624642 18378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:42:47.624884 18378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:42:47.627867 18378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60e5b015702e475c8f224342b163ad79: Sending DeleteTablet for 3 replicas of tablet 7176a90625e84eb8b476e7e8cbef0e9b
I20260812 09:42:47.628760 18536 tablet_service.cc:1514] Processing DeleteTablet for tablet 7176a90625e84eb8b476e7e8cbef0e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:47 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:45318
I20260812 09:42:47.629918 18828 tablet_service.cc:1514] Processing DeleteTablet for tablet 7176a90625e84eb8b476e7e8cbef0e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:47 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:50326
I20260812 09:42:47.631138 18687 tablet_service.cc:1514] Processing DeleteTablet for tablet 7176a90625e84eb8b476e7e8cbef0e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:47 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:42198
I20260812 09:42:47.632655 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 18425
I20260812 09:42:47.633559 18984 tablet_replica.cc:330] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: stopping tablet replica
I20260812 09:42:47.634049 18984 raft_consensus.cc:2229] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:42:47.634387 18984 raft_consensus.cc:2258] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:42:47.643123 18985 tablet_replica.cc:330] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: stopping tablet replica
I20260812 09:42:47.643564 18985 raft_consensus.cc:2229] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:42:47.643914 18985 raft_consensus.cc:2258] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:42:47.646082 18984 ts_tablet_manager.cc:1905] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:42:47.649883 18984 ts_tablet_manager.cc:1918] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:42:47.649998 18984 log.cc:1196] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/7176a90625e84eb8b476e7e8cbef0e9b
I20260812 09:42:47.650362 18984 ts_tablet_manager.cc:1939] T 7176a90625e84eb8b476e7e8cbef0e9b P c03ccf6afab442a284cef73a1fec15a0: Deleting consensus metadata
I20260812 09:42:47.651803 18362 catalog_manager.cc:4841] TS c03ccf6afab442a284cef73a1fec15a0 (127.3.147.3:42403): tablet 7176a90625e84eb8b476e7e8cbef0e9b (table test-table [id=dcddb7637e644f6d848b37886ac1cf44]) successfully deleted
I20260812 09:42:47.652015 18985 ts_tablet_manager.cc:1905] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:42:47.655191 18985 ts_tablet_manager.cc:1918] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:42:47.655345 18985 log.cc:1196] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/7176a90625e84eb8b476e7e8cbef0e9b
I20260812 09:42:47.655754 18985 ts_tablet_manager.cc:1939] T 7176a90625e84eb8b476e7e8cbef0e9b P 90ddea4924fd41e5884dbc762c94a696: Deleting consensus metadata
I20260812 09:42:47.659029 18365 catalog_manager.cc:4841] TS 90ddea4924fd41e5884dbc762c94a696 (127.3.147.2:42485): tablet 7176a90625e84eb8b476e7e8cbef0e9b (table test-table [id=dcddb7637e644f6d848b37886ac1cf44]) successfully deleted
W20260812 09:42:47.660014 18362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:41427 (error 108)
I20260812 09:42:47.660626 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 18619
I20260812 09:42:47.671190 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:47.670873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:41427 (local address 127.0.0.1:45332)
0812 09:42:47.671005 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:47.671082 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:41427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 09:42:47.671482 18362 catalog_manager.cc:4584] TS 6afc5512db52469ea667631d7c2bd1ee (127.3.147.1:41427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7176a90625e84eb8b476e7e8cbef0e9b: Network error: Client connection negotiation failed: client connection to 127.3.147.1:41427: connect: Connection refused (error 111)
I20260812 09:42:47.674101 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 18772
I20260812 09:42:47.689716 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 18343
I20260812 09:42:47.699167 3660 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 09:42:48.044817 3660 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 09:42:48.044968 3660 mini_ranger.cc:66] Stopping Ranger...
I20260812 09:42:48.390969 3660 mini_ranger.cc:68] Stopped Ranger
2026-08-12 09:42:48.391 UTC [17305] LOG: received smart shutdown request
2026-08-12 09:42:48.395 UTC [17305] LOG: background worker "logical replication launcher" (PID 17318) exited with exit code 1
2026-08-12 09:42:48.396 UTC [17310] LOG: shutting down
2026-08-12 09:42:48.573 UTC [17305] LOG: database system is shut down
2026-08-12T09:42:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50414 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:42:48 dist-test-slave-cbbz krb5kdc[18998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:42:48 dist-test-slave-cbbz krb5kdc[18998](info): set up 2 sockets
Aug 12 09:42:48 dist-test-slave-cbbz krb5kdc[18998](info): commencing operation
krb5kdc: starting...
W20260812 09:42:49.824856 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:42:49 dist-test-slave-cbbz krb5kdc[18998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:42:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:42:49.951380 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43401
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:43401,127.3.147.61:45663,127.3.147.60:40959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:50.061142 19011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:50.061395 19011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:50.061434 19011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:50.065512 19011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:50.065567 19011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:50.065586 19011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:50.065603 19011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:50.065618 19011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:50.070930 19011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:43401,127.3.147.61:45663,127.3.147.60:40959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:50.071949 19011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:50.072997 19011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:50.081491 19020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:50.081624 19011 server_base.cc:1034] running on GCE node
W20260812 09:42:50.081493 19017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:50.081492 19018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:50.082496 19011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:50.083566 19011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:50.084738 19011 hybrid_clock.cc:648] HybridClock initialized: now 1786527770084717 us; error 30 us; skew 500 ppm
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:50.087544 19011 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:42:50.088464 19011 webserver.cc:460] Webserver started at http://127.3.147.62:46167/ using document root <none> and password file <none>
I20260812 09:42:50.089031 19011 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:50.089093 19011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:50.089349 19011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:42:50.091351 19011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "2f9c173038fb43a7ba523655089324fb"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "06fbab73e174ce692f4e9661a8df52a5"
server_key_iv: "f30c6f4e48be170b3230137dadbbc904"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.091940 19011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "2f9c173038fb43a7ba523655089324fb"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "06fbab73e174ce692f4e9661a8df52a5"
server_key_iv: "f30c6f4e48be170b3230137dadbbc904"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.095659 19011 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:42:50.098104 19028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:50.099283 19011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 09:42:50.099434 19011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "2f9c173038fb43a7ba523655089324fb"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "06fbab73e174ce692f4e9661a8df52a5"
server_key_iv: "f30c6f4e48be170b3230137dadbbc904"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.099546 19011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:50.115988 19011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:50.119000 19011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:50.119210 19011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:50.126113 19011 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:43401
I20260812 09:42:50.126139 19093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:43401 every 8 connection(s)
I20260812 09:42:50.127111 19011 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:42:50.127905 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 19011
I20260812 09:42:50.128067 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:42:50.128474 3660 external_mini_cluster.cc:1428] Setting key 2cd18159cb5ee4430564bc4b82f5788f
I20260812 09:42:50.130695 19094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:50.134580 19094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.147.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.61@KRBTEST.COM" created.
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:50.150820 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.136964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36054 (local address 127.3.147.62:43401)
0812 09:42:50.137402 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.137420 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.137647 (+ 227us) server_negotiation.cc:407] Connection header received
0812 09:42:50.138586 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.138590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.138771 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.139034 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.139812 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.141196 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.142204 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.142835 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.143018 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.145717 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.145723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.145734 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.145754 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.147621 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.148096 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.148101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.148103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.148250 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.148566 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.148569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.148571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.148945 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.149106 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.149480 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.149742 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":123,"threads_started":1}
I20260812 09:42:50.150816 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.136964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43401 (local address 127.0.0.1:36054)
0812 09:42:50.137500 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.137537 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.138317 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.139014 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.139019 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.139358 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.139601 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.139614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.141329 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.141332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.142078 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.142087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.143008 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.143015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.143174 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.143816 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.143841 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.145558 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.147742 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.147752 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.147762 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.147985 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.148356 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.148359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.148361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.148468 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.149079 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.149086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.149236 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.149418 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.149822 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":117,"spinlock_wait_cycles":1408,"thread_start_us":194,"threads_started":1}
I20260812 09:42:50.153442 19094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:42:50.154299 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.153983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53278)
0812 09:42:50.154119 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.154233 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 09:42:50.154520 19029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
I20260812 09:42:50.155395 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.155171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53284)
0812 09:42:50.155295 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.155340 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 09:42:50.155560 19094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:45663: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab.
Entry for principal kudu/127.3.147.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab.
I20260812 09:42:50.158767 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.61:45663
--webserver_interface=127.3.147.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab
--principal=kudu/127.3.147.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:43401,127.3.147.61:45663,127.3.147.60:40959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:42:50.204843 19094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } attempt: 1
I20260812 09:42:50.205782 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.205465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53288)
0812 09:42:50.205632 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.205701 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 09:42:50.206521 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.206291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53294)
0812 09:42:50.206404 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.206458 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 09:42:50.206686 19094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:45663: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
W20260812 09:42:50.273883 19102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:50.274194 19102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:50.274271 19102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 09:42:50.274849 19094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } attempt: 2
I20260812 09:42:50.275743 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.275424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53308)
0812 09:42:50.275585 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.275664 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 09:42:50.276566 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.276373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53312)
0812 09:42:50.276471 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.276514 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 09:42:50.276710 19094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:45663: Network error: Client connection negotiation failed: client connection to 127.3.147.61:45663: connect: Connection refused (error 111)
W20260812 09:42:50.278987 19102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:50.279036 19102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:50.279057 19102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:50.279073 19102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:50.279089 19102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:50.284354 19102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:43401,127.3.147.61:45663,127.3.147.60:40959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab
--principal=kudu/127.3.147.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.61:45663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.147.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:50.285432 19102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:50.286406 19102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:50.294894 19110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:50.294879 19108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:50.294880 19107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:50.295289 19102 server_base.cc:1034] running on GCE node
I20260812 09:42:50.295882 19102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:50.296998 19102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:50.298187 19102 hybrid_clock.cc:648] HybridClock initialized: now 1786527770298180 us; error 40 us; skew 500 ppm
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:50.300972 19102 init.cc:377] Logged in from keytab as kudu/127.3.147.61@KRBTEST.COM (short username kudu)
I20260812 09:42:50.301798 19102 webserver.cc:460] Webserver started at http://127.3.147.61:40625/ using document root <none> and password file <none>
I20260812 09:42:50.302430 19102 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:50.302524 19102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:50.302829 19102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:42:50.305241 19102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/data/instance:
uuid: "864c33e20b9d4322b9232c0c40971170"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "50c63e8ad04951fe5186e2bb1ee57b42"
server_key_iv: "c13f327f99cdb9a27eeea3f81969bed2"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.305833 19102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/wal/instance:
uuid: "864c33e20b9d4322b9232c0c40971170"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "50c63e8ad04951fe5186e2bb1ee57b42"
server_key_iv: "c13f327f99cdb9a27eeea3f81969bed2"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.309593 19102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:42:50.311954 19118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:50.313196 19102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:42:50.313347 19102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/wal
uuid: "864c33e20b9d4322b9232c0c40971170"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "50c63e8ad04951fe5186e2bb1ee57b42"
server_key_iv: "c13f327f99cdb9a27eeea3f81969bed2"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.313458 19102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:50.333433 19102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:50.336402 19102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:50.336613 19102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:50.343842 19102 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.61:45663
I20260812 09:42:50.343935 19178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.61:45663 every 8 connection(s)
I20260812 09:42:50.344872 19102 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
I20260812 09:42:50.345579 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 19102
I20260812 09:42:50.345718 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-1/wal/instance
I20260812 09:42:50.346056 3660 external_mini_cluster.cc:1428] Setting key 7aec14a0fa637bd47bacc89134cf5168
I20260812 09:42:50.348297 19179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:50.352028 19179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.147.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.60@KRBTEST.COM" created.
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:50.367381 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.354029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.3.147.62:43401)
0812 09:42:50.354218 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.354224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.355267 (+ 1043us) server_negotiation.cc:407] Connection header received
0812 09:42:50.356261 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.356266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.356402 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.356559 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.358083 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.358762 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.360332 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.360750 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.360889 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.363393 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.363399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.363406 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.363442 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.365224 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.365745 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.365751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.365756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.365881 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.366262 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.366267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.366269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.366493 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.366615 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.367092 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.367201 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 09:42:50.367884 19094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } attempt: 3
I20260812 09:42:50.368330 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.354566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43401 (local address 127.0.0.1:36062)
0812 09:42:50.355114 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.355158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.355971 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.356730 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.356736 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.357107 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.357839 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.357872 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.358900 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.358907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.360198 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.360211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.360905 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.360908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.361045 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.361703 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.361719 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.363213 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.365352 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.365357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.365368 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.365630 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.365992 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.365995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.365996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.366151 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.366619 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.366624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.366855 (+ 231us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.367012 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.367291 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":150,"threads_started":1}
I20260812 09:42:50.369999 19179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab.
Entry for principal kudu/127.3.147.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab.
I20260812 09:42:50.376570 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.60:40959
--webserver_interface=127.3.147.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab
--principal=kudu/127.3.147.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:34331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:43401,127.3.147.61:45663,127.3.147.60:40959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:42:50.379014 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.370496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53330)
0812 09:42:50.370721 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.370735 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.370808 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.371061 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.371064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.371241 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.371319 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.371324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.372268 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.372270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.372749 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.372757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.373227 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.373231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.373338 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.373761 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.373771 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.375155 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.377428 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.377434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.377436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.377640 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.377945 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.377948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.377950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.378031 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.378479 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.378482 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.378564 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.378698 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.378826 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260812 09:42:50.379060 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.370552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.3.147.61:45663)
0812 09:42:50.370857 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.370866 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.370879 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:42:50.370907 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.370909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.370950 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.371049 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.371429 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.372069 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.372874 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.373119 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.373226 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.375325 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.375328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.375330 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.375415 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.377315 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.377753 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.377758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.377760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.377843 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.378119 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.378124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.378125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.378369 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.378514 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.378772 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.378903 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":114,"threads_started":1}
I20260812 09:42:50.381453 19179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:42:50.381866 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.368356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53322)
0812 09:42:50.368574 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.368588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.368660 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.369252 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.369255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.369429 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.369955 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.369963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.371135 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.371142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.371867 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.371877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.372530 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.372534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.372655 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.373176 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.373190 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.375146 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.380169 (+ 5023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.380172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.380175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.380400 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.380699 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.380702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.380704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.380787 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.381173 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.381178 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.381302 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.381417 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.381578 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 09:42:50.382241 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.381915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59608)
0812 09:42:50.382099 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.382172 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260812 09:42:50.382437 19124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
I20260812 09:42:50.382658 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.368475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53322 (local address 127.3.147.61:45663)
0812 09:42:50.368828 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.368851 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.368870 (+ 19us) server_negotiation.cc:407] Connection header received
0812 09:42:50.368905 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.368908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.369104 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.369245 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.370052 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.371011 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.371985 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.372413 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.372530 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.375276 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.375286 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.375300 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.375330 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.380022 (+ 4692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.380506 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.380512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.380515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.380598 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.380878 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.380882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.380884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.381049 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.381176 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.382220 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.382361 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":176,"threads_started":1}
I20260812 09:42:50.383268 19094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:42:50.383984 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.383757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59612)
0812 09:42:50.383887 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.383929 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 09:42:50.384753 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.384535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59636)
0812 09:42:50.384629 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.384689 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 09:42:50.384871 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.384513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59622)
0812 09:42:50.384735 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.384794 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260812 09:42:50.384920 19179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:40959: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
W20260812 09:42:50.385020 19094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:40959: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
I20260812 09:42:50.416270 19094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } attempt: 1
I20260812 09:42:50.417304 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.416945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59652)
0812 09:42:50.417150 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.417225 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260812 09:42:50.418176 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.417910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59662)
0812 09:42:50.418018 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.418070 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 09:42:50.418318 19094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:40959: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
I20260812 09:42:50.434177 19179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } attempt: 1
I20260812 09:42:50.435002 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.434729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59666)
0812 09:42:50.434871 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.434928 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 09:42:50.435706 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.435498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59678)
0812 09:42:50.435605 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.435655 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 09:42:50.435834 19179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:40959: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
I20260812 09:42:50.493474 19094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } attempt: 2
I20260812 09:42:50.494310 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.494001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59682)
0812 09:42:50.494165 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.494231 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 09:42:50.495101 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.494880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59692)
0812 09:42:50.494976 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.495020 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 09:42:50.495283 19094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:40959: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
W20260812 09:42:50.495227 19189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:50.495556 19189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:50.495669 19189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:50.499763 19189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:50.499823 19189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:50.499845 19189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:50.499892 19189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:50.499938 19189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:50.504006 19179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } attempt: 2
I20260812 09:42:50.504865 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.504567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59694)
0812 09:42:50.504721 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.504796 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 09:42:50.505633 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.505415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59708)
0812 09:42:50.505525 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.505570 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 09:42:50.505757 19179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:40959: Network error: Client connection negotiation failed: client connection to 127.3.147.60:40959: connect: Connection refused (error 111)
I20260812 09:42:50.505810 19189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:34331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:43401,127.3.147.61:45663,127.3.147.60:40959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab
--principal=kudu/127.3.147.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.60:40959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.147.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:50.506956 19189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:50.508036 19189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:50.516692 19198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:50.516741 19200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:50.516814 19196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:50.516999 19189 server_base.cc:1034] running on GCE node
I20260812 09:42:50.517562 19189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:50.518755 19189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:50.519917 19189 hybrid_clock.cc:648] HybridClock initialized: now 1786527770519903 us; error 31 us; skew 500 ppm
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:50.522867 19189 init.cc:377] Logged in from keytab as kudu/127.3.147.60@KRBTEST.COM (short username kudu)
I20260812 09:42:50.523818 19189 webserver.cc:460] Webserver started at http://127.3.147.60:42309/ using document root <none> and password file <none>
I20260812 09:42:50.524536 19189 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:50.524639 19189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:50.524879 19189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:42:50.526969 19189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/data/instance:
uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "c47f5c779caca6a6d3437263e3709761"
server_key_iv: "e118b033b54f18392a14d76bbf5d73e8"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.527568 19189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/wal/instance:
uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "c47f5c779caca6a6d3437263e3709761"
server_key_iv: "e118b033b54f18392a14d76bbf5d73e8"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.531457 19189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 09:42:50.534041 19208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:50.535344 19189 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 09:42:50.535496 19189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/wal
uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1"
format_stamp: "Formatted at 2026-08-12 09:42:50 on dist-test-slave-cbbz"
server_key: "c47f5c779caca6a6d3437263e3709761"
server_key_iv: "e118b033b54f18392a14d76bbf5d73e8"
server_key_version: "encryptionkey@0"
I20260812 09:42:50.535629 19189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:50.554394 19189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:50.557958 19189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:50.558184 19189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:50.566027 19189 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.60:40959
I20260812 09:42:50.566057 19267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.60:40959 every 8 connection(s)
I20260812 09:42:50.567016 19189 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
I20260812 09:42:50.569830 19268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:50.573653 19268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:42:50.575263 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 19189
I20260812 09:42:50.575415 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786527603645036-3660-0/minicluster-data/master-2/wal/instance
I20260812 09:42:50.575803 3660 external_mini_cluster.cc:1428] Setting key ee55765db6868c8cf9695849c95abd4b
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:42:50.588730 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.575692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36078 (local address 127.3.147.62:43401)
0812 09:42:50.575891 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.575896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.577156 (+ 1260us) server_negotiation.cc:407] Connection header received
0812 09:42:50.578029 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.578033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.578083 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.578180 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.579319 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.580173 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.581562 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.581929 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.582040 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.584800 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.584804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.584806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.584831 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.586495 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.587034 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.587038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.587040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.587124 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.587548 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.587553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.587554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.587769 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.587881 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.588420 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.588542 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 09:42:50.589577 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.576129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43401 (local address 127.0.0.1:36078)
0812 09:42:50.577018 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.577068 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.577789 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.578261 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.578266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.578756 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.579165 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.579179 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.580283 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.580286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.581413 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.581425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.582078 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.582081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.582242 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.582863 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.582890 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.584644 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.586621 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.586631 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.586641 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.586929 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.587226 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.587229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.587231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.587446 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.587881 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.587889 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.588112 (+ 223us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.588347 (+ 235us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.588662 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":1,"spinlock_wait_cycles":4864,"thread_start_us":162,"threads_started":1}
I20260812 09:42:50.589586 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.576535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59724 (local address 127.3.147.60:40959)
0812 09:42:50.576981 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.577009 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.577050 (+ 41us) server_negotiation.cc:407] Connection header received
0812 09:42:50.577773 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.577789 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.578048 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.578232 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.579174 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.580446 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.581562 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.582170 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.582249 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.585219 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.585225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.585235 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.585255 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.586852 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.587291 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.587296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.587297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.587406 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.587697 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.587701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.587703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.588123 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.588360 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.588580 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.588724 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"spinlock_wait_cycles":2304,"thread_start_us":148,"threads_started":1}
I20260812 09:42:50.590848 19268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:42:50.594455 19094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } attempt: 3
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:50.597680 19179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } attempt: 3
I20260812 09:42:50.600562 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.591493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.3.147.61:45663)
0812 09:42:50.591693 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.591697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.591709 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:42:50.591756 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.591758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.591834 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.591924 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.592407 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.592920 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.593704 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.593925 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.593999 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.596372 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.596375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.596378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.596402 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.598151 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.598628 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.598632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.598633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.598965 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.599534 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.599538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.599540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.599781 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.599878 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.600283 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.600404 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:42:50.601015 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.591320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:45663 (local address 127.0.0.1:53340)
0812 09:42:50.591440 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.591455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.591553 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.591974 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.591978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.592174 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.592302 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.592309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.593034 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.593040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.593604 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.593615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.594030 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.594034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.594134 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.594725 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.594735 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.596230 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.598278 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.598282 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.598283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.598512 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.599334 (+ 822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.599337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.599339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.599433 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.599926 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.599930 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.600038 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.600742 (+ 704us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.600866 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 09:42:50.601730 19268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:42:50.601846 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.592070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.3.147.62:43401)
0812 09:42:50.592211 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.592215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.592229 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:42:50.592297 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.592300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.592346 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.592419 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.593081 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.593523 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.594488 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.594859 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.594961 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.597719 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.597722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.597725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.597748 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.599622 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.600112 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.600118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.600120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.600241 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.600569 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.600574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.600579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.600800 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.601013 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.601563 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.601664 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:42:50.605919 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.594848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59732)
0812 09:42:50.594962 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.594976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.595050 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.595386 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.595389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.595582 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.595695 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.595706 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.596711 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.596714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.597554 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.597567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.598821 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.598827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.598922 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.599759 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.599774 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.601641 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.604111 (+ 2470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.604114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.604116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.604413 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.604827 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.604831 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.604833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.604923 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.605436 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.605439 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.605525 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.605630 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.605745 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 09:42:50.605952 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.595011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.3.147.60:40959)
0812 09:42:50.595118 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.595122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.595133 (+ 11us) server_negotiation.cc:407] Connection header received
0812 09:42:50.595165 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.595167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.595220 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.595356 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.595833 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.596566 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.598278 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.598670 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.598784 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.601787 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.601790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.601792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.601830 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.603980 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.604553 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.604558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.604561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.604648 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.605038 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.605044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.605046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.605301 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.605390 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.605709 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.605817 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:42:50.609131 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.598079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59742)
0812 09:42:50.598203 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.598218 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.598297 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.599307 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.599310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.599485 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.599594 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.599601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.601071 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.601078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.601850 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.601861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.602878 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.602884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.602988 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.603394 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.603410 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.604927 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.607407 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.607412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.607415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.607679 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.608017 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.608022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.608026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.608123 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.608593 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.608597 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.608695 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.608784 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.608918 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
Aug 12 09:42:50 dist-test-slave-cbbz krb5kdc[18998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
I20260812 09:42:50.609674 19273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.598341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.3.147.60:40959)
0812 09:42:50.599041 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.599057 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.599070 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:42:50.599108 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.599112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.599153 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.599280 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.599720 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.600940 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.601993 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.602385 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.603220 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.605065 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.605068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.605070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.605112 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.607264 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.607802 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.607805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.607808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.607902 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.608262 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.608267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.608270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.608482 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.609310 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.609392 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.609498 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"spinlock_wait_cycles":384,"thread_start_us":107,"threads_started":1}
I20260812 09:42:50.610008 19094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: Bootstrap starting.
I20260812 09:42:50.612828 19179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: Bootstrap starting.
I20260812 09:42:50.613209 19094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:50.613667 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.602228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:40959 (local address 127.0.0.1:59748)
0812 09:42:50.602333 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:50.602346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:50.602416 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:50.603424 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:50.603427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:50.603938 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:50.604035 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.604043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.605157 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.605162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.606254 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:50.606265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.607122 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.607126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:50.607216 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.607866 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:50.607895 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:50.609825 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:50.612285 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.612288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.612289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.612512 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.612801 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:50.612804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:50.612805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:50.612887 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:50.613262 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:50.613265 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:50.613336 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 09:42:50.613411 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 09:42:50.613522 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"spinlock_wait_cycles":6144}
I20260812 09:42:50.613833 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.603614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.3.147.61:45663)
0812 09:42:50.603717 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.603721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.603860 (+ 139us) server_negotiation.cc:407] Connection header received
0812 09:42:50.604104 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.604107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.604169 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.604222 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.605112 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.605776 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.606843 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.607135 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.607207 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.610051 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.610054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.610057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.610082 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.611767 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.612278 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.612282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.612283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.612419 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.612764 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.612767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.612769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.612963 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.613122 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.613581 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.613697 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 09:42:50.614178 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.602433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.3.147.60:40959)
0812 09:42:50.602734 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.602742 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.602755 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:42:50.602787 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.602790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.602831 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.603923 (+ 1092us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.604268 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.605057 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.606620 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.606991 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.609936 (+ 2945us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.610031 (+ 95us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.610037 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.610042 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.610065 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.612175 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.612632 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.612635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.612636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.612712 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.612984 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.612987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.612988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.613164 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.613817 (+ 653us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.613894 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.614009 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":125,"threads_started":1}
I20260812 09:42:50.614158 19094 log.cc:824] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:50.615274 19179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:50.615957 19179 log.cc:824] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:50.617328 19094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: No bootstrap required, opened a new log
I20260812 09:42:50.618741 19268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1: Bootstrap starting.
I20260812 09:42:50.620422 19094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:50.620675 19094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:50.620723 19268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:50.620745 19094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9c173038fb43a7ba523655089324fb, State: Initialized, Role: FOLLOWER
I20260812 09:42:50.621253 19094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:50.621497 19268 log.cc:824] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:50.622058 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:42:50.622186 19277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:50.622311 19277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:42:50.622680 19094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:50.623867 19268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1: No bootstrap required, opened a new log
I20260812 09:42:50.629037 19268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:50.629352 19268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:50.629444 19268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae198cc025e4fb09bbf9f865a1dd7b1, State: Initialized, Role: FOLLOWER
I20260812 09:42:50.630043 19268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:50.631469 19268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:50.627489 19011 master_runner.cc:186] Error getting master registration for 127.3.147.61:45663: OK, instance_id { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" instance_seqno: 1786527770341749 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 09:42:50.631601 19011 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 09:42:50.633675 19179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: No bootstrap required, opened a new log
I20260812 09:42:50.637781 19179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:50.638015 19179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:50.638067 19179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864c33e20b9d4322b9232c0c40971170, State: Initialized, Role: FOLLOWER
I20260812 09:42:50.638545 19179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:50.639950 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.625028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.3.147.61:45663)
0812 09:42:50.625269 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.625273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.625498 (+ 225us) server_negotiation.cc:407] Connection header received
0812 09:42:50.625680 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.625685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.625735 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.625809 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.626698 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.627376 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.628623 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.628970 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.629050 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.631524 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.631528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.631531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.631556 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.633332 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.637119 (+ 3787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.637125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.637128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.637217 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.637982 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.637987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.637990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.638215 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.638318 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.639594 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.639738 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260812 09:42:50.640472 19179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:50.641247 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.624606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36094 (local address 127.3.147.62:43401)
0812 09:42:50.624716 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.624724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.624946 (+ 222us) server_negotiation.cc:407] Connection header received
0812 09:42:50.625163 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.625167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.625217 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.625269 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.626336 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.627114 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.629855 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.630209 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.630300 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.634030 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.634034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.634037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.634062 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.636084 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.637570 (+ 1486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.637574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.637576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.637664 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.638642 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.638645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.638647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.639046 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.639211 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.640885 (+ 1674us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.641026 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 09:42:50.641927 19312 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:42:50.642005 19312 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:42:50.642196 19301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:50.642333 19301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 09:42:50.647398 19329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:42:50.647481 19329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:42:50.649403 19273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:50.625139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.3.147.60:40959)
0812 09:42:50.625239 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:50.625244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:50.628566 (+ 3322us) server_negotiation.cc:407] Connection header received
0812 09:42:50.628765 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:50.628768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:50.628810 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:50.628858 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:50.630867 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.631696 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.633757 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:50.634125 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:50.634201 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:50.635964 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:50.635969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:50.635974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:50.636002 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:50.647324 (+ 11322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.647891 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.647895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.647897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.647985 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:50.648360 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:50.648363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:50.648365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:50.648583 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:50.648711 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:50.649119 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 09:42:50.649257 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 09:42:50.637584 19189 master_runner.cc:186] Error getting master registration for 127.3.147.61:45663: OK, instance_id { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" instance_seqno: 1786527770341749 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 09:42:50.650033 19189 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 09:42:50.651418 19102 master_runner.cc:263] No leader master found from master 864c33e20b9d4322b9232c0c40971170
I20260812 09:42:50.651496 19102 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 09:42:50.651557 19300 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:42:50.651638 19300 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:42:50.651894 19313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:50.652019 19313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 09:42:50.809674 19216 tablet.cc:2369] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:42:50.838227 19126 tablet.cc:2369] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:42:50.871121 19034 tablet.cc:2369] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:42:51.030141 19301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:42:51.030383 19301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:51.031486 19301 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9c173038fb43a7ba523655089324fb (127.3.147.62:43401), 864c33e20b9d4322b9232c0c40971170 (127.3.147.61:45663)
I20260812 09:42:51.032047 19061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9c173038fb43a7ba523655089324fb" is_pre_election: true
I20260812 09:42:51.032040 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864c33e20b9d4322b9232c0c40971170" is_pre_election: true
I20260812 09:42:51.032385 19061 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ae198cc025e4fb09bbf9f865a1dd7b1 in term 0.
I20260812 09:42:51.032428 19151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ae198cc025e4fb09bbf9f865a1dd7b1 in term 0.
I20260812 09:42:51.032797 19211 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9c173038fb43a7ba523655089324fb, 6ae198cc025e4fb09bbf9f865a1dd7b1; no voters:
I20260812 09:42:51.033154 19301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:42:51.033262 19301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:42:51.033324 19301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:51.034430 19301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:51.034752 19301 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9c173038fb43a7ba523655089324fb (127.3.147.62:43401), 864c33e20b9d4322b9232c0c40971170 (127.3.147.61:45663)
I20260812 09:42:51.035087 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864c33e20b9d4322b9232c0c40971170"
I20260812 09:42:51.035087 19061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9c173038fb43a7ba523655089324fb"
I20260812 09:42:51.035212 19151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:51.035212 19061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:51.036530 19061 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae198cc025e4fb09bbf9f865a1dd7b1 in term 1.
I20260812 09:42:51.036530 19151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae198cc025e4fb09bbf9f865a1dd7b1 in term 1.
I20260812 09:42:51.036808 19211 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9c173038fb43a7ba523655089324fb, 6ae198cc025e4fb09bbf9f865a1dd7b1; no voters:
I20260812 09:42:51.036973 19301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:42:51.037179 19301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae198cc025e4fb09bbf9f865a1dd7b1, State: Running, Role: LEADER
I20260812 09:42:51.037519 19301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } }
I20260812 09:42:51.039433 19332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae198cc025e4fb09bbf9f865a1dd7b1. Latest consensus state: current_term: 1 leader_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:51.039549 19332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:51.039942 19334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:42:51.040954 19334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:42:51.045668 19151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [term 1 FOLLOWER]: Refusing update from remote peer 6ae198cc025e4fb09bbf9f865a1dd7b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:42:51.045702 19061 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [term 1 FOLLOWER]: Refusing update from remote peer 6ae198cc025e4fb09bbf9f865a1dd7b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:42:51.046352 19332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:42:51.046532 19301 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:42:51.050311 19277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae198cc025e4fb09bbf9f865a1dd7b1. Latest consensus state: current_term: 1 leader_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:51.050438 19277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:42:51.050433 19313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae198cc025e4fb09bbf9f865a1dd7b1. Latest consensus state: current_term: 1 leader_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:51.050539 19313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:42:51.052064 19335 mvcc.cc:205] Tried to move back new op lower bound from 7317617750196224000 to 7317617750172303360. Current Snapshot: MvccSnapshot[applied={T|T < 7317617750196224000}]
I20260812 09:42:51.052352 19332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:51.052479 19332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:51.052446 19301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:51.052521 19301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1 [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:51.053244 19313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:51.053360 19313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:42:51.053561 19277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9c173038fb43a7ba523655089324fb" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43401 } } peers { permanent_uuid: "864c33e20b9d4322b9232c0c40971170" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 45663 } } peers { permanent_uuid: "6ae198cc025e4fb09bbf9f865a1dd7b1" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 40959 } } }
I20260812 09:42:51.053596 19336 mvcc.cc:205] Tried to move back new op lower bound from 7317617750196224000 to 7317617750172303360. Current Snapshot: MvccSnapshot[applied={T|T < 7317617750196224000}]
I20260812 09:42:51.053681 19277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:42:51.054255 19334 catalog_manager.cc:1347] Generated new cluster ID: 4665354d991a4528be6067862c49063b
I20260812 09:42:51.054317 19334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:42:51.069588 19334 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:42:51.070729 19334 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:42:51.083784 19334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ae198cc025e4fb09bbf9f865a1dd7b1: Generated new TSK 0
I20260812 09:42:51.084645 19334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:42:51.648919 19329 catalog_manager.cc:1259] Loaded cluster ID: 4665354d991a4528be6067862c49063b
I20260812 09:42:51.649012 19329 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: loading cluster ID for follower catalog manager: success
I20260812 09:42:51.650763 19329 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: acquiring CA information for follower catalog manager: success
I20260812 09:42:51.652235 19329 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 864c33e20b9d4322b9232c0c40971170: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 09:42:51.653024 19300 catalog_manager.cc:1259] Loaded cluster ID: 4665354d991a4528be6067862c49063b
I20260812 09:42:51.653107 19300 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: loading cluster ID for follower catalog manager: success
I20260812 09:42:51.654901 19300 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: acquiring CA information for follower catalog manager: success
I20260812 09:42:51.655951 19300 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2f9c173038fb43a7ba523655089324fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 09:42:52.169476 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 19011
I20260812 09:42:52.178499 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 19102
I20260812 09:42:52.186255 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 19189
2026-08-12T09:42:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3480 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:42:52 dist-test-slave-cbbz krb5kdc[19353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:42:52 dist-test-slave-cbbz krb5kdc[19353](info): set up 2 sockets
Aug 12 09:42:52 dist-test-slave-cbbz krb5kdc[19353](info): commencing operation
krb5kdc: starting...
W20260812 09:42:53.292371 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:42:53 dist-test-slave-cbbz krb5kdc[19353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:42:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:42:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:42:53.417245 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43323
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:43323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:53.525943 19369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:53.526261 19369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:53.526376 19369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:53.530285 19369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:53.530368 19369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:53.530408 19369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:53.530467 19369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:53.530507 19369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:53.535615 19369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:43323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:53.536679 19369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:53.537717 19369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:53.546353 19375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:53.546365 19376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:53.546382 19378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:53.546679 19369 server_base.cc:1034] running on GCE node
I20260812 09:42:53.547281 19369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:53.549026 19369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:53.550280 19369 hybrid_clock.cc:648] HybridClock initialized: now 1786527773550254 us; error 41 us; skew 500 ppm
Aug 12 09:42:53 dist-test-slave-cbbz krb5kdc[19353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527773, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:53.553157 19369 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:42:53.553998 19369 webserver.cc:460] Webserver started at http://127.3.147.62:44391/ using document root <none> and password file <none>
I20260812 09:42:53.554538 19369 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:53.554639 19369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:53.554872 19369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:42:53.556912 19369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "3e066d982d454d77a8b81db42e99b0ac"
format_stamp: "Formatted at 2026-08-12 09:42:53 on dist-test-slave-cbbz"
server_key: "0095fcceb10aafab5bd18e22b9a90fda"
server_key_iv: "0d7d99e641191274797418f18816c485"
server_key_version: "encryptionkey@0"
I20260812 09:42:53.557487 19369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "3e066d982d454d77a8b81db42e99b0ac"
format_stamp: "Formatted at 2026-08-12 09:42:53 on dist-test-slave-cbbz"
server_key: "0095fcceb10aafab5bd18e22b9a90fda"
server_key_iv: "0d7d99e641191274797418f18816c485"
server_key_version: "encryptionkey@0"
I20260812 09:42:53.561141 19369 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:42:53.563475 19385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:53.564677 19369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:42:53.564812 19369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "3e066d982d454d77a8b81db42e99b0ac"
format_stamp: "Formatted at 2026-08-12 09:42:53 on dist-test-slave-cbbz"
server_key: "0095fcceb10aafab5bd18e22b9a90fda"
server_key_iv: "0d7d99e641191274797418f18816c485"
server_key_version: "encryptionkey@0"
I20260812 09:42:53.564920 19369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:53.576388 19369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:53.579170 19369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:53.579381 19369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:53.586575 19369 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:43323
I20260812 09:42:53.586669 19451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:43323 every 8 connection(s)
I20260812 09:42:53.587505 19369 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:42:53.590303 19452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:53.593116 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 19369
I20260812 09:42:53.593289 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:42:53.593573 3660 external_mini_cluster.cc:1428] Setting key 2abfd6e49b20858171fba408938325f0
I20260812 09:42:53.596335 19452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac: Bootstrap starting.
I20260812 09:42:53.598582 19452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:53.599360 19452 log.cc:824] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:53.601504 19452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac: No bootstrap required, opened a new log
Aug 12 09:42:53 dist-test-slave-cbbz krb5kdc[19353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:53.606454 19452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e066d982d454d77a8b81db42e99b0ac" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43323 } }
I20260812 09:42:53.606738 19452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:53.606863 19452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e066d982d454d77a8b81db42e99b0ac, State: Initialized, Role: FOLLOWER
I20260812 09:42:53.607410 19452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e066d982d454d77a8b81db42e99b0ac" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43323 } }
I20260812 09:42:53.607600 19452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:42:53.607674 19452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:42:53.607764 19452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:53.608191 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:53.594841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49400 (local address 127.3.147.62:43323)
0812 09:42:53.595249 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:53.595265 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:53.595301 (+ 36us) server_negotiation.cc:407] Connection header received
0812 09:42:53.595886 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:53.595901 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:53.596276 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:53.596608 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:53.597425 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:53.598398 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:53.599329 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:53.599835 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:53.600009 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:53.602751 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:53.602758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:53.602774 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:53.602807 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:53.604937 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:53.605399 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:53.605406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:53.605412 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:53.605576 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:53.605852 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:53.605855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:53.605857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:53.606376 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:53.606607 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:53.606765 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 09:42:53.607091 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":134,"threads_started":1}
I20260812 09:42:53.608875 19452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e066d982d454d77a8b81db42e99b0ac" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43323 } }
I20260812 09:42:53.609272 19452 leader_election.cc:302] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e066d982d454d77a8b81db42e99b0ac; no voters:
I20260812 09:42:53.609627 19452 leader_election.cc:288] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:42:53.609750 19457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:42:53.609972 19457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [term 1 LEADER]: Becoming Leader. State: Replica: 3e066d982d454d77a8b81db42e99b0ac, State: Running, Role: LEADER
I20260812 09:42:53.610597 19457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e066d982d454d77a8b81db42e99b0ac" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43323 } }
I20260812 09:42:53.610983 19452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:53.611274 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:42:53.612339 19459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e066d982d454d77a8b81db42e99b0ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e066d982d454d77a8b81db42e99b0ac" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43323 } } }
I20260812 09:42:53.612488 19459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:53.612495 19460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e066d982d454d77a8b81db42e99b0ac. Latest consensus state: current_term: 1 leader_uuid: "3e066d982d454d77a8b81db42e99b0ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e066d982d454d77a8b81db42e99b0ac" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43323 } } }
I20260812 09:42:53.612598 19460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac [sys.catalog]: This master's current role is: LEADER
W20260812 09:42:53.616099 19474 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:42:53.616216 19474 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:42:53.616288 19467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:42:53.617241 19467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:42:53.622742 19467 catalog_manager.cc:1347] Generated new cluster ID: 4d0162d9c42841c69b79286309ed0106
I20260812 09:42:53.622812 19467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:42:53.640687 19467 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:42:53.641605 19467 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:42:53.647284 19467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e066d982d454d77a8b81db42e99b0ac: Generated new TSK 0
I20260812 09:42:53.647944 19467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:42:53.667426 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 19369
2026-08-12T09:42:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1481 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:42:53 dist-test-slave-cbbz krb5kdc[19481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:42:53 dist-test-slave-cbbz krb5kdc[19481](info): set up 2 sockets
Aug 12 09:42:53 dist-test-slave-cbbz krb5kdc[19481](info): commencing operation
krb5kdc: starting...
W20260812 09:42:54.782078 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:42:54 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:42:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:42:54.904831 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44477
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:44477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:55.016103 19499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:55.016373 19499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:55.016412 19499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:55.020423 19499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:55.020469 19499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:55.020489 19499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:55.020538 19499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:55.020570 19499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:55.025622 19499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:44477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:55.026631 19499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:55.027629 19499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:55.036353 19504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:55.036360 19505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:55.036360 19507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:55.036725 19499 server_base.cc:1034] running on GCE node
I20260812 09:42:55.037288 19499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:55.038338 19499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:55.039528 19499 hybrid_clock.cc:648] HybridClock initialized: now 1786527775039517 us; error 28 us; skew 500 ppm
Aug 12 09:42:55 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:55.046176 19499 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:42:55.047144 19499 webserver.cc:460] Webserver started at http://127.3.147.62:39247/ using document root <none> and password file <none>
I20260812 09:42:55.047757 19499 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:55.047848 19499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:55.048094 19499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:42:55.050207 19499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "51a141c10c9940fa975403010a5558bd"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "5bee367ff1418bc4763e55328789301b"
server_key_iv: "616a60d08f5d37220ccb59e20f8a7117"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.050792 19499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "51a141c10c9940fa975403010a5558bd"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "5bee367ff1418bc4763e55328789301b"
server_key_iv: "616a60d08f5d37220ccb59e20f8a7117"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.054689 19499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 09:42:55.057086 19515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.058274 19499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:42:55.058413 19499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "51a141c10c9940fa975403010a5558bd"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "5bee367ff1418bc4763e55328789301b"
server_key_iv: "616a60d08f5d37220ccb59e20f8a7117"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.058522 19499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:55.086751 19499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:55.089435 19499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:55.089632 19499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:55.096740 19499 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:44477
I20260812 09:42:55.097031 19571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:44477 every 8 connection(s)
I20260812 09:42:55.097739 19499 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:42:55.100544 19572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:55.100921 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 19499
I20260812 09:42:55.101058 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:42:55.101384 3660 external_mini_cluster.cc:1428] Setting key 71c41c55db6ba1ee5c147f18ada31a31
I20260812 09:42:55.107030 19572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Bootstrap starting.
I20260812 09:42:55.109377 19572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Neither blocks nor log segments found. Creating new log.
Aug 12 09:42:55 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:55.110294 19572 log.cc:824] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:55.112128 19572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: No bootstrap required, opened a new log
I20260812 09:42:55.115489 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.102779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.3.147.62:44477)
0812 09:42:55.103200 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.103214 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.103248 (+ 34us) server_negotiation.cc:407] Connection header received
0812 09:42:55.103906 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.103922 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.104320 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.104643 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:55.105484 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.106471 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.107415 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.107854 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.108022 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.110746 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:55.110751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:55.110763 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:55.110789 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:55.112689 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.113135 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.113141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.113145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.113252 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.113541 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.113545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.113547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.113977 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:55.114186 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.114500 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.114789 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":122,"threads_started":1}
I20260812 09:42:55.115846 19572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:55.116102 19572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:55.116226 19572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a141c10c9940fa975403010a5558bd, State: Initialized, Role: FOLLOWER
I20260812 09:42:55.116736 19572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:55.116894 19572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:42:55.116959 19572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:42:55.117053 19572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:55.118194 19572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:55.118484 19572 leader_election.cc:302] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51a141c10c9940fa975403010a5558bd; no voters:
I20260812 09:42:55.118726 19572 leader_election.cc:288] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:42:55.118846 19577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:42:55.119110 19577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 1 LEADER]: Becoming Leader. State: Replica: 51a141c10c9940fa975403010a5558bd, State: Running, Role: LEADER
I20260812 09:42:55.119445 19577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:55.119925 19572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:55.121104 19578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51a141c10c9940fa975403010a5558bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } } }
I20260812 09:42:55.121258 19578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:55.121578 19579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51a141c10c9940fa975403010a5558bd. Latest consensus state: current_term: 1 leader_uuid: "51a141c10c9940fa975403010a5558bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } } }
I20260812 09:42:55.121747 19579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:55.121663 19589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:42:55.125028 19589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:42:55.130733 19589 catalog_manager.cc:1347] Generated new cluster ID: 93b5d2d89e9d433f9d758248f5dad2d5
I20260812 09:42:55.130793 19589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:42:55.156654 19589 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:42:55.157951 19589 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:42:55.163319 19589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Generated new TSK 0
I20260812 09:42:55.164034 19589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:42:55.195118 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44477
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:55.308444 19605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:55.308741 19605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:55.308830 19605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:55.312898 19605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:55.312978 19605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:55.313064 19605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:42:55.318200 19605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:55.319209 19605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:55.320325 19605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:55.330474 19613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:55.330528 19611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:55.330498 19610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:55.330720 19605 server_base.cc:1034] running on GCE node
I20260812 09:42:55.331254 19605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:55.331854 19605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:55.333015 19605 hybrid_clock.cc:648] HybridClock initialized: now 1786527775332967 us; error 60 us; skew 500 ppm
Aug 12 09:42:55 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:55.335819 19605 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:42:55.336721 19605 webserver.cc:460] Webserver started at http://127.3.147.1:46353/ using document root <none> and password file <none>
I20260812 09:42:55.337276 19605 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:55.337365 19605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:55.337604 19605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:42:55.339543 19605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "7bd797339c5142cb8f148bd40c4e7266"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "8eb53062b1f0e6a6eebc7daee0e93341"
server_key_iv: "36fcf6447f7c7859c7b9dda957f14720"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.340108 19605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "7bd797339c5142cb8f148bd40c4e7266"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "8eb53062b1f0e6a6eebc7daee0e93341"
server_key_iv: "36fcf6447f7c7859c7b9dda957f14720"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.343672 19605 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:42:55.346084 19621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.347330 19605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:42:55.347515 19605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "7bd797339c5142cb8f148bd40c4e7266"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "8eb53062b1f0e6a6eebc7daee0e93341"
server_key_iv: "36fcf6447f7c7859c7b9dda957f14720"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.347647 19605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:55.364643 19605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:55.367826 19605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:55.368026 19605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:55.368680 19605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:55.369484 19605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:55.369556 19605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.369626 19605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:55.369673 19605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.378909 19605 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:45893
I20260812 09:42:55.378961 19749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:45893 every 8 connection(s)
I20260812 09:42:55.379833 19605 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:42:55.381477 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 19605
I20260812 09:42:55.381634 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:42:55.381981 3660 external_mini_cluster.cc:1428] Setting key a49f1a489bdacc8cc4965784cac3196b
Aug 12 09:42:55 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:42:55.395027 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.381748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:40675 (local address 127.3.147.62:44477)
0812 09:42:55.381919 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.381924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.383112 (+ 1188us) server_negotiation.cc:407] Connection header received
0812 09:42:55.384261 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.384264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.384307 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.384391 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:55.385911 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.386818 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.387915 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.388393 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.388541 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.391400 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:55.391403 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:55.391406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:55.391428 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:55.392982 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.393436 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.393438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.393439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.393509 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.393925 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.393929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.393931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.394114 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:55.394228 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.394731 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.394858 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:42:55.396173 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.382228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:40675)
0812 09:42:55.382941 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:55.382994 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:55.384002 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:55.384568 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:55.384580 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:55.385095 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:55.385762 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.385778 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.386942 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.386944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.387786 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.387794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.388561 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.388564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.388705 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.389375 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:55.389391 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:55.391228 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:55.393090 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:55.393095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:55.393105 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:55.393351 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:55.393605 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:55.393607 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:55.393609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:55.393816 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:55.394253 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:55.394259 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:55.394487 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 09:42:55.394677 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 09:42:55.395071 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":133,"threads_started":1}
I20260812 09:42:55.397588 19750 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44477
I20260812 09:42:55.397850 19750 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:55.398447 19750 heartbeater.cc:511] Master 127.3.147.62:44477 requested a full tablet report, sending...
I20260812 09:42:55.400275 19534 ts_manager.cc:194] Registered new tserver with Master: 7bd797339c5142cb8f148bd40c4e7266 (127.3.147.1:45893)
I20260812 09:42:55.401980 19534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:40675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:42:55.411221 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44477
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:55.521135 19758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:55.521379 19758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:55.521420 19758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:55.525429 19758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:55.525480 19758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:55.525552 19758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:42:55.530750 19758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:55.531626 19758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:55.532625 19758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:55.542030 19766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:55.542030 19763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:55.542040 19764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:55.542253 19758 server_base.cc:1034] running on GCE node
I20260812 09:42:55.542791 19758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:55.543390 19758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:55.544572 19758 hybrid_clock.cc:648] HybridClock initialized: now 1786527775544558 us; error 26 us; skew 500 ppm
Aug 12 09:42:55 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:55.547250 19758 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:42:55.548102 19758 webserver.cc:460] Webserver started at http://127.3.147.2:46855/ using document root <none> and password file <none>
I20260812 09:42:55.548707 19758 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:55.548804 19758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:55.549037 19758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:42:55.551096 19758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "f6f44c8ddecf42e5b36851b48164cf86"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "ec4b0a345cf2e1a9e3361042be5293f5"
server_key_iv: "69db21883d312525db09674e5a8d4921"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.551739 19758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "f6f44c8ddecf42e5b36851b48164cf86"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "ec4b0a345cf2e1a9e3361042be5293f5"
server_key_iv: "69db21883d312525db09674e5a8d4921"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.555670 19758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 09:42:55.558064 19775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.559259 19758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:42:55.559396 19758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "f6f44c8ddecf42e5b36851b48164cf86"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "ec4b0a345cf2e1a9e3361042be5293f5"
server_key_iv: "69db21883d312525db09674e5a8d4921"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.559523 19758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:55.570670 19758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:55.573838 19758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:55.574261 19758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:55.574868 19758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:55.575696 19758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:55.575770 19758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.575842 19758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:55.575886 19758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.585220 19758 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:37993
I20260812 09:42:55.585250 19906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:37993 every 8 connection(s)
I20260812 09:42:55.586127 19758 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:42:55.587193 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 19758
I20260812 09:42:55.587337 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:42:55.587695 3660 external_mini_cluster.cc:1428] Setting key c661201e76d8cb83c91c3a689478b9df
Aug 12 09:42:55 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:42:55.600839 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.588123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:58489 (local address 127.3.147.62:44477)
0812 09:42:55.588283 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.588302 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.589647 (+ 1345us) server_negotiation.cc:407] Connection header received
0812 09:42:55.591028 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.591034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.591092 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.591181 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:55.592556 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.593218 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.594340 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.594584 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.594701 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.597188 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:55.597192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:55.597195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:55.597220 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:55.598634 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.599220 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.599223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.599224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.599302 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.599657 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.599660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.599662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.599874 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:55.599988 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.600538 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.600665 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:42:55.601796 19910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.588643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:58489)
0812 09:42:55.589478 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:55.589536 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:55.590775 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:55.591382 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:55.591393 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:55.591913 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:55.592409 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.592422 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.593333 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.593335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.594206 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.594215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.594753 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.594757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.594925 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.595528 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:55.595544 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:55.597007 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:55.598759 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:55.598765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:55.598779 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:55.599096 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:55.599397 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:55.599401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:55.599403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:55.599550 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:55.599987 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:55.599993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:55.600298 (+ 305us) client_negotiation.cc:769] Sending connection context
0812 09:42:55.600479 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 09:42:55.600805 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":144,"threads_started":1}
I20260812 09:42:55.603111 19907 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44477
I20260812 09:42:55.603364 19907 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:55.603986 19907 heartbeater.cc:511] Master 127.3.147.62:44477 requested a full tablet report, sending...
I20260812 09:42:55.605176 19534 ts_manager.cc:194] Registered new tserver with Master: f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993)
I20260812 09:42:55.606029 19534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:58489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:42:55.620267 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44477
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:55.736833 19913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:55.737192 19913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:55.737268 19913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:55.744338 19913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:55.744441 19913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:55.744621 19913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:42:55.752264 19913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:55.753273 19913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:55.754246 19913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:55.764108 19918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:55.764108 19923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:55.764513 19913 server_base.cc:1034] running on GCE node
W20260812 09:42:55.764587 19920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:55.765018 19913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:55.765633 19913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:55.766832 19913 hybrid_clock.cc:648] HybridClock initialized: now 1786527775766818 us; error 25 us; skew 500 ppm
Aug 12 09:42:55 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:55.769624 19913 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:42:55.770442 19913 webserver.cc:460] Webserver started at http://127.3.147.3:43955/ using document root <none> and password file <none>
I20260812 09:42:55.771000 19913 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:55.771092 19913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:55.771342 19913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:42:55.773425 19913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "5d973305767c40f5b9a261e2e8c223c8"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "1915d992dd5df268ac653d8381f8624c"
server_key_iv: "bbec778fa0d946c6e90a7a019d5318a1"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.773975 19913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "5d973305767c40f5b9a261e2e8c223c8"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "1915d992dd5df268ac653d8381f8624c"
server_key_iv: "bbec778fa0d946c6e90a7a019d5318a1"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.777690 19913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:42:55.780009 19931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.781256 19913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:42:55.781395 19913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "5d973305767c40f5b9a261e2e8c223c8"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "1915d992dd5df268ac653d8381f8624c"
server_key_iv: "bbec778fa0d946c6e90a7a019d5318a1"
server_key_version: "encryptionkey@0"
I20260812 09:42:55.781502 19913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:55.791698 19913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:55.794364 19913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:55.794559 19913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:55.795096 19913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:55.795902 19913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:55.796000 19913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.796088 19913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:55.796141 19913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:55.805219 19913 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42273
I20260812 09:42:55.805298 20060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42273 every 8 connection(s)
I20260812 09:42:55.806069 19913 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:42:55.807302 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 19913
I20260812 09:42:55.807426 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:42:55.807695 3660 external_mini_cluster.cc:1428] Setting key 333ff3b8f777d842864f17a9abd24866
Aug 12 09:42:55 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:55.819492 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.807874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:42033 (local address 127.3.147.62:44477)
0812 09:42:55.808037 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.808042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.809393 (+ 1351us) server_negotiation.cc:407] Connection header received
0812 09:42:55.810448 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.810451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.810505 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.810608 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:55.811740 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.812331 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.813445 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.813666 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.813776 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.816175 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:55.816187 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:55.816190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:55.816221 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:55.817423 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.817977 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.817982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.817984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.818067 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.818412 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.818417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.818418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.818600 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:55.818709 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.819200 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.819322 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 09:42:55.820490 20064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.808362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:42033)
0812 09:42:55.809233 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:55.809291 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:55.810256 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:55.810791 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:55.810798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:55.811135 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:55.811591 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.811604 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.812457 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.812461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.813318 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.813328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.813827 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.813832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.813999 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.814530 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:55.814546 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:55.815967 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:55.817546 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:55.817552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:55.817565 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:55.817869 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:55.818162 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:55.818166 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:55.818168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:55.818313 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:55.818719 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:55.818728 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:55.818952 (+ 224us) client_negotiation.cc:769] Sending connection context
0812 09:42:55.819147 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 09:42:55.819459 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":174,"threads_started":1}
I20260812 09:42:55.821622 20061 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44477
I20260812 09:42:55.821923 20061 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:55.822417 20061 heartbeater.cc:511] Master 127.3.147.62:44477 requested a full tablet report, sending...
I20260812 09:42:55.823421 19534 ts_manager.cc:194] Registered new tserver with Master: 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273)
I20260812 09:42:55.823985 19534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:42033
I20260812 09:42:55.833642 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:42:55.842792 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.835577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.3.147.62:44477)
0812 09:42:55.835690 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.835694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.835897 (+ 203us) server_negotiation.cc:407] Connection header received
0812 09:42:55.835991 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.835993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.836028 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.836092 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:55.836782 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.837267 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.838042 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.838246 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.838319 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.839744 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:55.839747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:55.839748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:55.839765 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:55.841052 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.841496 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.841498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.841499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.841581 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:55.841884 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:55.841888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:55.841889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:55.842202 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:55.842332 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.842550 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.842669 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 09:42:55.845570 19534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:42:55.848630 19534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:42:55.864040 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.857191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42273 (local address 127.0.0.1:34804)
0812 09:42:55.857589 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:55.857650 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:55.857807 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:55.858802 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:55.858808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:55.858829 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:55.858945 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.858950 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.860661 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.860668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.862086 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.862097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.863428 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.863432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.863578 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.863622 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 09:42:55.863746 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 09:42:55.863886 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260812 09:42:55.864378 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.857972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.3.147.1:45893)
0812 09:42:55.859391 (+ 1419us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.859408 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.859428 (+ 20us) server_negotiation.cc:407] Connection header received
0812 09:42:55.859503 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.859508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.859682 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.859835 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:55.860111 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.861292 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.862854 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.863431 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.863913 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.864059 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.864111 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.864228 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1306,"thread_start_us":71,"threads_started":1}
I20260812 09:42:55.864415 20075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.857859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:37993 (local address 127.0.0.1:44564)
0812 09:42:55.858496 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:55.858510 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:55.858643 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:55.859140 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:55.859144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:55.859165 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:55.859283 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.859291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.860968 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.860972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.862440 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.862451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.863901 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.863905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.864007 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.864046 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 09:42:55.864139 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 09:42:55.864242 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":17,"thread_start_us":97,"threads_started":1}
I20260812 09:42:55.864895 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.857552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:45893 (local address 127.0.0.1:55354)
0812 09:42:55.858158 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:55.858189 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:55.858293 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:55.859850 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:55.859854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:55.859876 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:55.859969 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.859976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.861442 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.861445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.862749 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.862760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.863579 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.863582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.863653 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.863748 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 09:42:55.863793 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 09:42:55.864773 (+ 980us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":927,"thread_start_us":110,"threads_started":1}
I20260812 09:42:55.864924 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.858304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.3.147.2:37993)
0812 09:42:55.858747 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.858760 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.858782 (+ 22us) server_negotiation.cc:407] Connection header received
0812 09:42:55.858860 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.858864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.859004 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.859122 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:55.859448 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.860648 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.862597 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.863512 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.863665 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.863825 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.864613 (+ 788us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.864724 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":103,"threads_started":1}
I20260812 09:42:55.864120 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.857365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.3.147.3:42273)
0812 09:42:55.858171 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.858190 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.858223 (+ 33us) server_negotiation.cc:407] Connection header received
0812 09:42:55.858320 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.858331 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.858577 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.859150 (+ 573us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:55.859340 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.860509 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.862235 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.863279 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.863421 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.863616 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.863784 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.863902 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":138,"threads_started":1}
I20260812 09:42:55.867724 19833 tablet_service.cc:1467] Processing CreateTablet for tablet b4c953b68a6d43db8cb2f8f849486daf (DEFAULT_TABLE table=test-table [id=7dcf191b073c411489ab03b530cacc7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:55.867794 19991 tablet_service.cc:1467] Processing CreateTablet for tablet b4c953b68a6d43db8cb2f8f849486daf (DEFAULT_TABLE table=test-table [id=7dcf191b073c411489ab03b530cacc7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:55.867794 19676 tablet_service.cc:1467] Processing CreateTablet for tablet b4c953b68a6d43db8cb2f8f849486daf (DEFAULT_TABLE table=test-table [id=7dcf191b073c411489ab03b530cacc7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:55.868980 19676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c953b68a6d43db8cb2f8f849486daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:55.868990 19991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c953b68a6d43db8cb2f8f849486daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:55.868983 19833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c953b68a6d43db8cb2f8f849486daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:55.876067 20080 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: Bootstrap starting.
I20260812 09:42:55.876942 20083 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: Bootstrap starting.
I20260812 09:42:55.877338 20081 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Bootstrap starting.
I20260812 09:42:55.879046 20080 tablet_bootstrap.cc:654] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:55.879094 20081 tablet_bootstrap.cc:654] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:55.879174 20083 tablet_bootstrap.cc:654] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:55.880362 20081 log.cc:824] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:55.880388 20080 log.cc:824] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:55.880388 20083 log.cc:824] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:55.882215 20081 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: No bootstrap required, opened a new log
I20260812 09:42:55.882378 20083 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: No bootstrap required, opened a new log
I20260812 09:42:55.882375 20080 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: No bootstrap required, opened a new log
I20260812 09:42:55.882419 20081 ts_tablet_manager.cc:1397] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:42:55.882561 20083 ts_tablet_manager.cc:1397] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 09:42:55.882561 20080 ts_tablet_manager.cc:1397] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 09:42:55.885608 20081 raft_consensus.cc:348] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.885985 20081 raft_consensus.cc:374] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:55.886097 20081 raft_consensus.cc:729] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd797339c5142cb8f148bd40c4e7266, State: Initialized, Role: FOLLOWER
I20260812 09:42:55.886432 20083 raft_consensus.cc:348] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.886428 20080 raft_consensus.cc:348] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.886715 20080 raft_consensus.cc:374] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:55.886752 20083 raft_consensus.cc:374] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:55.886685 20081 consensus_queue.cc:260] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.886799 20080 raft_consensus.cc:729] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f44c8ddecf42e5b36851b48164cf86, State: Initialized, Role: FOLLOWER
I20260812 09:42:55.886813 20083 raft_consensus.cc:729] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d973305767c40f5b9a261e2e8c223c8, State: Initialized, Role: FOLLOWER
I20260812 09:42:55.887265 20083 consensus_queue.cc:260] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.887398 19750 heartbeater.cc:503] Master 127.3.147.62:44477 was elected leader, sending a full tablet report...
I20260812 09:42:55.887920 20061 heartbeater.cc:503] Master 127.3.147.62:44477 was elected leader, sending a full tablet report...
I20260812 09:42:55.888247 20083 ts_tablet_manager.cc:1428] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 09:42:55.888288 20080 consensus_queue.cc:260] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.888573 20081 ts_tablet_manager.cc:1428] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:42:55.889164 19907 heartbeater.cc:503] Master 127.3.147.62:44477 was elected leader, sending a full tablet report...
I20260812 09:42:55.889256 20080 ts_tablet_manager.cc:1428] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260812 09:42:55.929888 20087 raft_consensus.cc:482] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:42:55.930089 20087 raft_consensus.cc:504] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.931228 20087 leader_election.cc:288] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273), f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993)
I20260812 09:42:55.935588 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.931699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:49731 (local address 127.3.147.3:42273)
0812 09:42:55.931900 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.931907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.931934 (+ 27us) server_negotiation.cc:407] Connection header received
0812 09:42:55.932169 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.932176 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.932255 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.932342 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:55.932616 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.933228 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.934560 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.934973 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.935146 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.935238 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.935380 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.935459 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 09:42:55.935629 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.931510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42273 (local address 127.3.147.1:49731)
0812 09:42:55.931864 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:55.931878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:55.932000 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:55.932353 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:55.932356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:55.932375 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:55.932501 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.932506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.933342 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.933349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.934436 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.934446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.935142 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.935146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.935247 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.935275 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 09:42:55.935357 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 09:42:55.935434 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":143,"threads_started":1}
I20260812 09:42:55.936015 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.932021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:37993 (local address 127.3.147.1:40717)
0812 09:42:55.932323 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:55.932336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:55.932442 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:55.932724 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:55.932727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:55.932743 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:55.932861 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.932866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.933750 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.933755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.934685 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:55.934694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.935514 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.935518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:55.935631 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.935661 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 09:42:55.935806 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 09:42:55.935882 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":126,"threads_started":1}
I20260812 09:42:55.936013 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.932221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:40717 (local address 127.3.147.2:37993)
0812 09:42:55.932360 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.932366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.932389 (+ 23us) server_negotiation.cc:407] Connection header received
0812 09:42:55.932557 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.932560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.932615 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.932704 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:55.932992 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.933638 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.934818 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.935395 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.935490 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.935548 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.935831 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.935906 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 09:42:55.936471 20012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c953b68a6d43db8cb2f8f849486daf" candidate_uuid: "7bd797339c5142cb8f148bd40c4e7266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d973305767c40f5b9a261e2e8c223c8" is_pre_election: true
I20260812 09:42:55.936807 19855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c953b68a6d43db8cb2f8f849486daf" candidate_uuid: "7bd797339c5142cb8f148bd40c4e7266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f44c8ddecf42e5b36851b48164cf86" is_pre_election: true
I20260812 09:42:55.936902 20012 raft_consensus.cc:2454] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bd797339c5142cb8f148bd40c4e7266 in term 0.
I20260812 09:42:55.937112 19855 raft_consensus.cc:2454] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bd797339c5142cb8f148bd40c4e7266 in term 0.
I20260812 09:42:55.937479 19622 leader_election.cc:302] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d973305767c40f5b9a261e2e8c223c8, 7bd797339c5142cb8f148bd40c4e7266; no voters:
I20260812 09:42:55.937831 20087 raft_consensus.cc:2781] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:42:55.937927 20087 raft_consensus.cc:482] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:42:55.937974 20087 raft_consensus.cc:3037] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:55.939078 20087 raft_consensus.cc:504] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.939458 20087 leader_election.cc:288] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 1 election: Requested vote from peers 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273), f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993)
I20260812 09:42:55.939800 20012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c953b68a6d43db8cb2f8f849486daf" candidate_uuid: "7bd797339c5142cb8f148bd40c4e7266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d973305767c40f5b9a261e2e8c223c8"
I20260812 09:42:55.939857 19855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c953b68a6d43db8cb2f8f849486daf" candidate_uuid: "7bd797339c5142cb8f148bd40c4e7266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f44c8ddecf42e5b36851b48164cf86"
I20260812 09:42:55.939944 20012 raft_consensus.cc:3037] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:55.939970 19855 raft_consensus.cc:3037] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:55.941326 20012 raft_consensus.cc:2454] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bd797339c5142cb8f148bd40c4e7266 in term 1.
I20260812 09:42:55.941326 19855 raft_consensus.cc:2454] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bd797339c5142cb8f148bd40c4e7266 in term 1.
I20260812 09:42:55.941658 19622 leader_election.cc:302] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bd797339c5142cb8f148bd40c4e7266, f6f44c8ddecf42e5b36851b48164cf86; no voters:
I20260812 09:42:55.941869 20087 raft_consensus.cc:2781] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:42:55.942103 20087 raft_consensus.cc:686] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 1 LEADER]: Becoming Leader. State: Replica: 7bd797339c5142cb8f148bd40c4e7266, State: Running, Role: LEADER
I20260812 09:42:55.942441 20087 consensus_queue.cc:237] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:55.945963 19533 catalog_manager.cc:5495] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bd797339c5142cb8f148bd40c4e7266 (127.3.147.1). New cstate: current_term: 1 leader_uuid: "7bd797339c5142cb8f148bd40c4e7266" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } health_report { overall_health: HEALTHY } } }
I20260812 09:42:55.962062 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:55.958293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55364 (local address 127.3.147.1:45893)
0812 09:42:55.958449 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:55.958454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:55.958469 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:42:55.958617 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:55.958620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:55.958689 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:55.958807 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:42:55.959069 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.959585 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.960553 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:55.960779 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:55.960895 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:55.961217 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:42:55.961625 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:42:55.961770 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:55.961824 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 09:42:55.961904 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 09:42:55.968601 19855 raft_consensus.cc:1261] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 1 FOLLOWER]: Refusing update from remote peer 7bd797339c5142cb8f148bd40c4e7266: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:42:55.968601 20012 raft_consensus.cc:1261] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 1 FOLLOWER]: Refusing update from remote peer 7bd797339c5142cb8f148bd40c4e7266: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:42:55.969419 20087 consensus_queue.cc:1035] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:42:55.969691 20092 consensus_queue.cc:1035] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:42:55.988515 19534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:42:55.988723 19534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:42:55.991402 19534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Sending DeleteTablet for 3 replicas of tablet b4c953b68a6d43db8cb2f8f849486daf
I20260812 09:42:55.992331 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 19605
I20260812 09:42:55.992390 19833 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c953b68a6d43db8cb2f8f849486daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:55 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:44564
I20260812 09:42:55.993073 19991 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c953b68a6d43db8cb2f8f849486daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:55 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:34804
I20260812 09:42:55.993865 20107 tablet_replica.cc:330] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: stopping tablet replica
I20260812 09:42:55.994191 20107 raft_consensus.cc:2229] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:42:55.994485 20107 raft_consensus.cc:2258] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:42:55.995201 20106 tablet_replica.cc:330] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: stopping tablet replica
I20260812 09:42:55.995639 20106 raft_consensus.cc:2229] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:42:55.996055 20106 raft_consensus.cc:2258] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:42:55.998040 20107 ts_tablet_manager.cc:1905] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:42:55.998068 20106 ts_tablet_manager.cc:1905] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:42:56.000603 20106 ts_tablet_manager.cc:1918] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:42:56.000743 20106 log.cc:1196] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/b4c953b68a6d43db8cb2f8f849486daf
I20260812 09:42:56.001070 20106 ts_tablet_manager.cc:1939] T b4c953b68a6d43db8cb2f8f849486daf P f6f44c8ddecf42e5b36851b48164cf86: Deleting consensus metadata
I20260812 09:42:56.001874 20107 ts_tablet_manager.cc:1918] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:42:56.002004 20107 log.cc:1196] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/b4c953b68a6d43db8cb2f8f849486daf
I20260812 09:42:56.002389 19517 catalog_manager.cc:4841] TS f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993): tablet b4c953b68a6d43db8cb2f8f849486daf (table test-table [id=7dcf191b073c411489ab03b530cacc7f]) successfully deleted
I20260812 09:42:56.002460 20107 ts_tablet_manager.cc:1939] T b4c953b68a6d43db8cb2f8f849486daf P 5d973305767c40f5b9a261e2e8c223c8: Deleting consensus metadata
I20260812 09:42:56.003790 19517 catalog_manager.cc:4841] TS 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273): tablet b4c953b68a6d43db8cb2f8f849486daf (table test-table [id=7dcf191b073c411489ab03b530cacc7f]) successfully deleted
W20260812 09:42:56.006331 19520 connection.cc:535] client connection to 127.3.147.1:45893 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 09:42:56.006492 19520 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 09:42:56.006747 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 19758
I20260812 09:42:56.009575 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.009160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:45893 (local address 127.0.0.1:55380)
0812 09:42:56.009352 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:56.009447 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 09:42:56.009788 19520 catalog_manager.cc:4584] TS 7bd797339c5142cb8f148bd40c4e7266 (127.3.147.1:45893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4c953b68a6d43db8cb2f8f849486daf: Network error: Client connection negotiation failed: client connection to 127.3.147.1:45893: connect: Connection refused (error 111)
I20260812 09:42:56.016078 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 19913
I20260812 09:42:56.024744 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 19499
I20260812 09:42:56.034430 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44477
--webserver_interface=127.3.147.62
--webserver_port=39247
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:44477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:56.146498 20109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:56.146751 20109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:56.146790 20109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:56.150748 20109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:56.150810 20109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:56.150828 20109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:56.150844 20109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:56.150858 20109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:56.156032 20109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:44477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=39247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:56.156975 20109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:56.157917 20109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:56.173164 20117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:56.173205 20116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:56.173375 20109 server_base.cc:1034] running on GCE node
W20260812 09:42:56.173205 20119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:56.174181 20109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:56.175376 20109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:56.176594 20109 hybrid_clock.cc:648] HybridClock initialized: now 1786527776176567 us; error 40 us; skew 500 ppm
Aug 12 09:42:56 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:56.182569 20109 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:42:56.183485 20109 webserver.cc:460] Webserver started at http://127.3.147.62:39247/ using document root <none> and password file <none>
I20260812 09:42:56.184088 20109 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:56.184255 20109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:56.190156 20109 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 09:42:56.193269 20127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:56.194855 20109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:42:56.194994 20109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "51a141c10c9940fa975403010a5558bd"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "5bee367ff1418bc4763e55328789301b"
server_key_iv: "616a60d08f5d37220ccb59e20f8a7117"
server_key_version: "encryptionkey@0"
I20260812 09:42:56.195847 20109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:56.215593 20109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:56.218914 20109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:56.219138 20109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:56.227607 20109 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:44477
I20260812 09:42:56.227962 20188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:44477 every 8 connection(s)
I20260812 09:42:56.229135 20109 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:42:56.232388 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 20109
I20260812 09:42:56.232844 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:45893
--local_ip_for_outbound_sockets=127.3.147.1
--tserver_master_addrs=127.3.147.62:44477
--webserver_port=46353
--webserver_interface=127.3.147.1
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:42:56.235688 20189 sys_catalog.cc:263] Verifying existing consensus state
I20260812 09:42:56.238404 20189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Bootstrap starting.
I20260812 09:42:56.263610 20189 log.cc:824] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:56.274861 20189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:42:56.275327 20189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Bootstrap complete.
I20260812 09:42:56.281136 20189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:56.281569 20189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a141c10c9940fa975403010a5558bd, State: Initialized, Role: FOLLOWER
I20260812 09:42:56.282156 20189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:56.282307 20189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:42:56.282408 20189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:42:56.282532 20189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 1 FOLLOWER]: Advancing to term 2
I20260812 09:42:56.284618 20189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:56.284965 20189 leader_election.cc:302] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51a141c10c9940fa975403010a5558bd; no voters:
I20260812 09:42:56.285336 20189 leader_election.cc:288] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 09:42:56.285478 20194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 2 FOLLOWER]: Leader election won for term 2
I20260812 09:42:56.285764 20194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 2 LEADER]: Becoming Leader. State: Replica: 51a141c10c9940fa975403010a5558bd, State: Running, Role: LEADER
I20260812 09:42:56.286069 20194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:56.286775 20189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:56.287075 20197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "51a141c10c9940fa975403010a5558bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } } }
I20260812 09:42:56.287204 20197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:56.287580 20199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:42:56.287559 20196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51a141c10c9940fa975403010a5558bd. Latest consensus state: current_term: 2 leader_uuid: "51a141c10c9940fa975403010a5558bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } } }
I20260812 09:42:56.287813 20196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:56.292716 20199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:42:56.293500 20199 catalog_manager.cc:1259] Loaded cluster ID: 93b5d2d89e9d433f9d758248f5dad2d5
I20260812 09:42:56.293563 20199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:42:56.294986 20199 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 09:42:56.299669 20199 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 09:42:56.305090 20199 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:42:56.305892 20199 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Loaded TSK: 0
I20260812 09:42:56.307102 20199 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 09:42:56.369673 20192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:56.369940 20192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:56.370018 20192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:56.374043 20192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:56.374123 20192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:56.374235 20192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:42:56.379405 20192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:45893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=46353
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:56.380499 20192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:56.381521 20192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:56.392442 20222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:56.392489 20225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:56.392467 20223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:56.392999 20192 server_base.cc:1034] running on GCE node
I20260812 09:42:56.393414 20192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:56.394078 20192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:56.395257 20192 hybrid_clock.cc:648] HybridClock initialized: now 1786527776395225 us; error 51 us; skew 500 ppm
Aug 12 09:42:56 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:56.398041 20192 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:42:56.399050 20192 webserver.cc:460] Webserver started at http://127.3.147.1:46353/ using document root <none> and password file <none>
I20260812 09:42:56.399627 20192 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:56.399705 20192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:56.403836 20192 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:42:56.406098 20234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:56.407425 20192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 09:42:56.407521 20192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "7bd797339c5142cb8f148bd40c4e7266"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "8eb53062b1f0e6a6eebc7daee0e93341"
server_key_iv: "36fcf6447f7c7859c7b9dda957f14720"
server_key_version: "encryptionkey@0"
I20260812 09:42:56.408071 20192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:56.426852 20192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:56.430047 20192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:56.430266 20192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:56.430953 20192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:56.432052 20241 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 09:42:56.437098 20192 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 09:42:56.437189 20192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 09:42:56.437280 20192 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 09:42:56.438966 20192 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 09:42:56.439038 20192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 09:42:56.439321 20241 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Bootstrap starting.
I20260812 09:42:56.450117 20192 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:45893
I20260812 09:42:56.450196 20364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:45893 every 8 connection(s)
I20260812 09:42:56.451081 20192 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:42:56.451359 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 20192
I20260812 09:42:56.451874 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:37993
--local_ip_for_outbound_sockets=127.3.147.2
--tserver_master_addrs=127.3.147.62:44477
--webserver_port=46855
--webserver_interface=127.3.147.2
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:42:56.460382 20241 log.cc:824] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:42:56 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:56.468478 20241 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:42:56.468876 20241 tablet_bootstrap.cc:492] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Bootstrap complete.
I20260812 09:42:56.470331 20241 ts_tablet_manager.cc:1397] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Time spent bootstrapping tablet: real 0.031s user 0.022s sys 0.007s
I20260812 09:42:56.472636 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.453444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:47131 (local address 127.3.147.62:44477)
0812 09:42:56.455986 (+ 2542us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:56.456008 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:56.456060 (+ 52us) server_negotiation.cc:407] Connection header received
0812 09:42:56.456509 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:56.456530 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:56.456951 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:56.457362 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:56.458991 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.460491 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.462477 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.463195 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.463372 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.466651 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:56.466659 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:56.466676 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:56.466711 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:56.468882 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.469532 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.469542 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.469550 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.469728 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.470131 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.470137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.470142 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.470613 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:56.470751 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:56.471466 (+ 715us) server_negotiation.cc:299] Negotiation successful
0812 09:42:56.471769 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2348,"thread_start_us":124,"threads_started":1}
I20260812 09:42:56.472643 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.453453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:47131)
0812 09:42:56.454021 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:56.454078 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:56.454931 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:56.457447 (+ 2516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:56.457460 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:56.458029 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:56.458765 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.458791 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.460676 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.460683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.462291 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.462308 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.463369 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.463375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.463611 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.464407 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:56.464436 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:56.466434 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:56.469027 (+ 2593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:56.469033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:56.469046 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:56.469397 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:56.469849 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:56.469854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:56.469858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:56.470029 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:56.470822 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:56.470828 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:56.471119 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 09:42:56.471323 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 09:42:56.471653 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":154,"threads_started":1}
I20260812 09:42:56.473364 20241 raft_consensus.cc:348] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:56.473827 20241 raft_consensus.cc:729] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd797339c5142cb8f148bd40c4e7266, State: Initialized, Role: FOLLOWER
I20260812 09:42:56.474315 20241 consensus_queue.cc:260] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:56.475241 20241 ts_tablet_manager.cc:1428] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260812 09:42:56.475934 20366 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44477
I20260812 09:42:56.476213 20366 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:56.476758 20366 heartbeater.cc:511] Master 127.3.147.62:44477 requested a full tablet report, sending...
I20260812 09:42:56.479301 20150 ts_manager.cc:194] Registered new tserver with Master: 7bd797339c5142cb8f148bd40c4e7266 (127.3.147.1:45893)
I20260812 09:42:56.482692 20150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:47131
I20260812 09:42:56.485303 20366 heartbeater.cc:503] Master 127.3.147.62:44477 was elected leader, sending a full tablet report...
Aug 12 09:42:56 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:42:56.492604 20374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.482136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:45893 (local address 127.0.0.1:55388)
0812 09:42:56.482870 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:56.482912 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:56.483050 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:56.483584 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:56.483590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:56.483959 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:56.484063 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.484070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.485042 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.485048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.485823 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.485835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.486416 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.486420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.486551 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.487145 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:56.487157 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:56.488849 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:56.490782 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:56.490791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:56.490796 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:56.491037 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:56.491403 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:56.491407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:56.491409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:56.491496 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:56.491936 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:56.491942 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:56.492067 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 09:42:56.492235 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 09:42:56.492426 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":113,"threads_started":1}
I20260812 09:42:56.492637 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.482236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.3.147.1:45893)
0812 09:42:56.482582 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:56.482593 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:56.483015 (+ 422us) server_negotiation.cc:407] Connection header received
0812 09:42:56.483212 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:56.483218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:56.483412 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:56.483545 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:56.484226 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.484922 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.485941 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.486274 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.486402 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.489009 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:56.489021 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:56.489036 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:56.489058 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:56.490654 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.491212 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.491224 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.491226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.491303 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.491588 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.491592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.491594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.491820 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:56.491962 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:56.492329 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 09:42:56.492491 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":99,"threads_started":1}
I20260812 09:42:56.493654 20288 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c953b68a6d43db8cb2f8f849486daf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:55 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:55388
I20260812 09:42:56.494637 20377 tablet_replica.cc:330] stopping tablet replica
I20260812 09:42:56.494927 20377 raft_consensus.cc:2229] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:42:56.495167 20377 raft_consensus.cc:2258] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:42:56.496114 20377 ts_tablet_manager.cc:1905] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:42:56.499832 20377 ts_tablet_manager.cc:1918] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:42:56.499948 20377 log.cc:1196] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/b4c953b68a6d43db8cb2f8f849486daf
I20260812 09:42:56.500267 20377 ts_tablet_manager.cc:1939] T b4c953b68a6d43db8cb2f8f849486daf P 7bd797339c5142cb8f148bd40c4e7266: Deleting consensus metadata
I20260812 09:42:56.502079 20130 catalog_manager.cc:4841] TS 7bd797339c5142cb8f148bd40c4e7266 (127.3.147.1:45893): tablet b4c953b68a6d43db8cb2f8f849486daf (table test-table [id=7dcf191b073c411489ab03b530cacc7f]) successfully deleted
W20260812 09:42:56.576423 20369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:56.576679 20369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:56.576722 20369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:56.580856 20369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:56.580919 20369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:56.580979 20369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:42:56.586406 20369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:37993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=46855
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:56.587720 20369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:56.588800 20369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:56.598445 20384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:56.598503 20386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:56.598430 20383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:56.598912 20369 server_base.cc:1034] running on GCE node
I20260812 09:42:56.599277 20369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:56.599908 20369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:56.601094 20369 hybrid_clock.cc:648] HybridClock initialized: now 1786527776601091 us; error 80 us; skew 500 ppm
Aug 12 09:42:56 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:56.603840 20369 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:42:56.604713 20369 webserver.cc:460] Webserver started at http://127.3.147.2:46855/ using document root <none> and password file <none>
I20260812 09:42:56.605258 20369 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:56.605361 20369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:56.609165 20369 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:42:56.611268 20395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:56.612632 20369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:42:56.612733 20369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "f6f44c8ddecf42e5b36851b48164cf86"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "ec4b0a345cf2e1a9e3361042be5293f5"
server_key_iv: "69db21883d312525db09674e5a8d4921"
server_key_version: "encryptionkey@0"
I20260812 09:42:56.613250 20369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:56.623193 20369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:56.625975 20369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:56.626129 20369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:56.626636 20369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:56.627409 20369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:56.627455 20369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:56.627494 20369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:56.627516 20369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:56.637036 20369 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:37993
I20260812 09:42:56.637131 20538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:37993 every 8 connection(s)
I20260812 09:42:56.638085 20369 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:42:56.640934 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 20369
I20260812 09:42:56.641644 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:42273
--local_ip_for_outbound_sockets=127.3.147.3
--tserver_master_addrs=127.3.147.62:44477
--webserver_port=43955
--webserver_interface=127.3.147.3
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:42:56 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:56.657359 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.640054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:41435 (local address 127.3.147.62:44477)
0812 09:42:56.640285 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:56.640290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:56.644443 (+ 4153us) server_negotiation.cc:407] Connection header received
0812 09:42:56.645763 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:56.645771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:56.645865 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:56.645996 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:56.647780 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.648802 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.650349 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.650607 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.650731 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.653592 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:56.653602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:56.653604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:56.653630 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:56.655083 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.655604 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.655611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.655613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.655707 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.656061 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.656065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.656067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.656316 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:56.656427 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:56.657034 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 09:42:56.657175 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260812 09:42:56.658329 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.640515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:41435)
0812 09:42:56.644223 (+ 3708us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:56.644307 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:56.645494 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:56.646231 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:56.646257 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:56.646851 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:56.647595 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.647616 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.648953 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.648956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.650215 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.650226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.650778 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.650782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.650964 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.651630 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:56.651646 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:56.653412 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:56.655210 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:56.655216 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:56.655230 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:56.655495 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:56.655805 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:56.655809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:56.655811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:56.655960 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:56.656426 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:56.656433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:56.656681 (+ 248us) client_negotiation.cc:769] Sending connection context
0812 09:42:56.656992 (+ 311us) client_negotiation.cc:240] Negotiation successful
0812 09:42:56.657333 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3425,"thread_start_us":140,"threads_started":1}
I20260812 09:42:56.659576 20539 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44477
I20260812 09:42:56.659847 20539 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:56.660471 20539 heartbeater.cc:511] Master 127.3.147.62:44477 requested a full tablet report, sending...
I20260812 09:42:56.661687 20150 ts_manager.cc:194] Registered new tserver with Master: f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993)
I20260812 09:42:56.662278 20150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:41435
W20260812 09:42:56.761046 20543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:56.761319 20543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:56.761391 20543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:56.765411 20543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:56.765486 20543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:56.765570 20543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:42:56.770859 20543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:42273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=43955
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:56.771865 20543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:56.772913 20543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:56.782573 20551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:56.782634 20549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:56.782644 20548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:56.782727 20543 server_base.cc:1034] running on GCE node
I20260812 09:42:56.783259 20543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:56.783849 20543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:56.785008 20543 hybrid_clock.cc:648] HybridClock initialized: now 1786527776784958 us; error 61 us; skew 500 ppm
Aug 12 09:42:56 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:56.787855 20543 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:42:56.788787 20543 webserver.cc:460] Webserver started at http://127.3.147.3:43955/ using document root <none> and password file <none>
I20260812 09:42:56.789404 20543 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:56.789525 20543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:56.793388 20543 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:42:56.795552 20558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:56.796838 20543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:42:56.796983 20543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "5d973305767c40f5b9a261e2e8c223c8"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "1915d992dd5df268ac653d8381f8624c"
server_key_iv: "bbec778fa0d946c6e90a7a019d5318a1"
server_key_version: "encryptionkey@0"
I20260812 09:42:56.797541 20543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:56.821262 20543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:56.824321 20543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:56.824528 20543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:56.825168 20543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:56.826037 20543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:56.826110 20543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:56.826184 20543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:56.826237 20543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:56.835717 20543 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42273
I20260812 09:42:56.835772 20689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42273 every 8 connection(s)
I20260812 09:42:56.836897 20543 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:42:56.840413 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 20543
Aug 12 09:42:56 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:42:56.849880 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.841131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.3.147.62:44477)
0812 09:42:56.841394 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:56.841398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:56.841574 (+ 176us) server_negotiation.cc:407] Connection header received
0812 09:42:56.841749 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:56.841752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:56.841817 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:56.841915 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:56.842678 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.843255 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.844301 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.844527 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.844680 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.846236 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:56.846240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:56.846242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:56.846274 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:56.847774 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.848271 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.848275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.848277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.848420 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.848748 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.848751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.848752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.849047 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:56.849303 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:56.849476 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 09:42:56.849647 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":119,"threads_started":1}
I20260812 09:42:56.852044 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.838790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:53585 (local address 127.3.147.62:44477)
0812 09:42:56.838932 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:56.838936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:56.839871 (+ 935us) server_negotiation.cc:407] Connection header received
0812 09:42:56.841615 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:56.841619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:56.841685 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:56.841795 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:56.842992 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.843515 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.844795 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.845023 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.845143 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.847837 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:56.847846 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:56.847847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:56.847870 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:56.849644 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.850288 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.850293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.850296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.850388 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:56.850826 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:56.850830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:56.850832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:56.851069 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:56.851160 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:56.851745 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 09:42:56.851863 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 09:42:56.852850 20692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.839189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:53585)
0812 09:42:56.839700 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:56.839746 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:56.841358 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:56.841991 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:56.841998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:56.842375 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:56.842841 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.842854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.843668 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.843678 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.844655 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.844666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.845184 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.845189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.845360 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.845922 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:56.845939 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:56.847662 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:56.849786 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:56.849791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:56.849805 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:56.850171 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:56.850517 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:56.850520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:56.850522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:56.850705 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:56.851170 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:56.851176 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:56.851467 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 09:42:56.851680 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 09:42:56.852009 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":119,"threads_started":1}
I20260812 09:42:56.853899 20690 heartbeater.cc:347] Connected to a master server at 127.3.147.62:44477
I20260812 09:42:56.854156 20690 heartbeater.cc:464] Registering TS with master...
I20260812 09:42:56.854636 20690 heartbeater.cc:511] Master 127.3.147.62:44477 requested a full tablet report, sending...
I20260812 09:42:56.855625 20150 ts_manager.cc:194] Registered new tserver with Master: 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273)
I20260812 09:42:56.856240 20150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:53585
I20260812 09:42:56.863478 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:42:56.867792 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.864055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.3.147.62:44477)
0812 09:42:56.864241 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:56.864246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:56.864425 (+ 179us) server_negotiation.cc:407] Connection header received
0812 09:42:56.864608 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:56.864611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:56.864711 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:56.864817 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:42:56.865064 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.865552 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.866440 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.866664 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.866774 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.867070 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:42:56.867432 (+ 362us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:42:56.867553 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:56.867613 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 09:42:56.867659 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 09:42:56.868443 20150 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:42:56.871584 20150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:42:56.879629 20288 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3cb7d3948c47acb806c1c332ef1c24 (DEFAULT_TABLE table=test-table [id=fa9057fcbc174a7da8840e6a38e533d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:56.880362 20288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3cb7d3948c47acb806c1c332ef1c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:56.883236 20241 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Bootstrap starting.
I20260812 09:42:56.884223 20241 tablet_bootstrap.cc:654] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:56.884223 20374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.878502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:37993 (local address 127.0.0.1:44580)
0812 09:42:56.878661 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:56.878676 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:56.878754 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:56.879628 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:56.879631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:56.879649 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:56.879771 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.879796 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.881559 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.881566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.882793 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.882804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.883835 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.883837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.883911 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.883945 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 09:42:56.883998 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 09:42:56.884057 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 09:42:56.884347 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.878690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44580 (local address 127.3.147.2:37993)
0812 09:42:56.879100 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:56.879116 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:56.879141 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:42:56.879225 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:56.879231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:56.879419 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:56.880036 (+ 617us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:56.880229 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.881408 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.882906 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.883719 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.883832 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.883975 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:56.884031 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 09:42:56.884186 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":159,"threads_started":1}
I20260812 09:42:56.885545 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.878984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42273 (local address 127.0.0.1:34810)
0812 09:42:56.879709 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:56.879724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:56.879872 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:56.880561 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:56.880564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:56.880581 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:56.880663 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.880670 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.881957 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.881962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.882721 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:56.882730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.885017 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.885022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:56.885141 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.885168 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 09:42:56.885258 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 09:42:56.885360 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260812 09:42:56.885581 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:56.879275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.3.147.3:42273)
0812 09:42:56.879675 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:56.879688 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:56.879828 (+ 140us) server_negotiation.cc:407] Connection header received
0812 09:42:56.880006 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:56.880011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:56.880241 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:56.880400 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:56.880796 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.881789 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.882853 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:56.884876 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:56.884983 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:56.885104 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:56.885318 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 09:42:56.885426 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":167,"threads_started":1}
I20260812 09:42:56.885774 20241 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: No bootstrap required, opened a new log
I20260812 09:42:56.885866 20241 ts_tablet_manager.cc:1397] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 09:42:56.886395 20241 raft_consensus.cc:348] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:56.886530 20241 raft_consensus.cc:374] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:56.886579 20241 raft_consensus.cc:729] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd797339c5142cb8f148bd40c4e7266, State: Initialized, Role: FOLLOWER
I20260812 09:42:56.886704 20241 consensus_queue.cc:260] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:56.886965 20241 ts_tablet_manager.cc:1428] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 09:42:56.887697 20454 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3cb7d3948c47acb806c1c332ef1c24 (DEFAULT_TABLE table=test-table [id=fa9057fcbc174a7da8840e6a38e533d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:56.888918 20454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3cb7d3948c47acb806c1c332ef1c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:56.888602 20615 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3cb7d3948c47acb806c1c332ef1c24 (DEFAULT_TABLE table=test-table [id=fa9057fcbc174a7da8840e6a38e533d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:42:56.889545 20615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3cb7d3948c47acb806c1c332ef1c24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:42:56.895416 20708 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Bootstrap starting.
I20260812 09:42:56.896121 20706 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Bootstrap starting.
I20260812 09:42:56.898155 20708 tablet_bootstrap.cc:654] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:56.898233 20706 tablet_bootstrap.cc:654] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Neither blocks nor log segments found. Creating new log.
I20260812 09:42:56.898923 20706 log.cc:824] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:56.898924 20708 log.cc:824] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:56.901109 20708 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: No bootstrap required, opened a new log
I20260812 09:42:56.901206 20706 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: No bootstrap required, opened a new log
I20260812 09:42:56.901341 20708 ts_tablet_manager.cc:1397] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 09:42:56.901418 20706 ts_tablet_manager.cc:1397] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:42:56.904312 20708 raft_consensus.cc:348] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:56.904645 20708 raft_consensus.cc:374] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:56.904469 20706 raft_consensus.cc:348] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:56.904724 20708 raft_consensus.cc:729] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f44c8ddecf42e5b36851b48164cf86, State: Initialized, Role: FOLLOWER
I20260812 09:42:56.904739 20706 raft_consensus.cc:374] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:42:56.904801 20706 raft_consensus.cc:729] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d973305767c40f5b9a261e2e8c223c8, State: Initialized, Role: FOLLOWER
I20260812 09:42:56.905184 20708 consensus_queue.cc:260] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:56.905275 20706 consensus_queue.cc:260] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:56.905833 20539 heartbeater.cc:503] Master 127.3.147.62:44477 was elected leader, sending a full tablet report...
I20260812 09:42:56.906138 20706 ts_tablet_manager.cc:1428] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:42:56.906191 20708 ts_tablet_manager.cc:1428] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 09:42:56.906472 20690 heartbeater.cc:503] Master 127.3.147.62:44477 was elected leader, sending a full tablet report...
W20260812 09:42:56.952741 20367 tablet.cc:2369] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:42:57.088220 20691 tablet.cc:2369] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:42:57.115037 20712 raft_consensus.cc:482] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:42:57.115264 20712 raft_consensus.cc:504] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:57.116552 20712 leader_election.cc:288] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993), 7bd797339c5142cb8f148bd40c4e7266 (127.3.147.1:45893)
I20260812 09:42:57.120391 20714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.116947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:60033 (local address 127.3.147.1:45893)
0812 09:42:57.117246 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:57.117249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:57.117261 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:42:57.117309 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:57.117311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:57.117363 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:57.117481 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:57.117799 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.118314 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.119360 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.119807 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.119932 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:57.120106 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:57.120149 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 09:42:57.120251 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":104,"threads_started":1}
I20260812 09:42:57.120388 20692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.116803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:45893 (local address 127.3.147.3:60033)
0812 09:42:57.116981 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.116998 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:57.117113 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:57.117492 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:57.117495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:57.117513 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:57.117640 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:57.117646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.118429 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.118435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:57.119224 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:57.119232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.119936 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.119940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:57.120047 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:57.120069 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 09:42:57.120171 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 09:42:57.120248 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 09:42:57.120612 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.116947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:57271 (local address 127.3.147.2:37993)
0812 09:42:57.117100 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:57.117106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:57.117394 (+ 288us) server_negotiation.cc:407] Connection header received
0812 09:42:57.117561 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:57.117564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:57.117610 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:57.117708 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:42:57.117987 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.118482 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.119557 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.119954 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.120070 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:57.120134 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:57.120396 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 09:42:57.120477 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 09:42:57.121057 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.116803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:37993 (local address 127.3.147.3:57271)
0812 09:42:57.117295 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.117307 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:57.117437 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:57.117719 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:57.117722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:57.117738 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:42:57.117872 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:57.117878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.118600 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.118603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:57.119425 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:57.119434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.120088 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.120092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:57.120246 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:57.120269 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:42:57.120823 (+ 554us) client_negotiation.cc:240] Negotiation successful
0812 09:42:57.120926 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":140,"thread_start_us":122,"threads_started":1}
I20260812 09:42:57.121284 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3cb7d3948c47acb806c1c332ef1c24" candidate_uuid: "5d973305767c40f5b9a261e2e8c223c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd797339c5142cb8f148bd40c4e7266" is_pre_election: true
I20260812 09:42:57.121625 20314 raft_consensus.cc:2454] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d973305767c40f5b9a261e2e8c223c8 in term 0.
I20260812 09:42:57.121691 20482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3cb7d3948c47acb806c1c332ef1c24" candidate_uuid: "5d973305767c40f5b9a261e2e8c223c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f44c8ddecf42e5b36851b48164cf86" is_pre_election: true
I20260812 09:42:57.122042 20482 raft_consensus.cc:2454] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d973305767c40f5b9a261e2e8c223c8 in term 0.
I20260812 09:42:57.122187 20562 leader_election.cc:302] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d973305767c40f5b9a261e2e8c223c8, 7bd797339c5142cb8f148bd40c4e7266; no voters:
I20260812 09:42:57.122514 20712 raft_consensus.cc:2781] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:42:57.122606 20712 raft_consensus.cc:482] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:42:57.122638 20712 raft_consensus.cc:3037] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:57.123729 20712 raft_consensus.cc:504] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:57.124118 20712 leader_election.cc:288] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [CANDIDATE]: Term 1 election: Requested vote from peers f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993), 7bd797339c5142cb8f148bd40c4e7266 (127.3.147.1:45893)
I20260812 09:42:57.124464 20482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3cb7d3948c47acb806c1c332ef1c24" candidate_uuid: "5d973305767c40f5b9a261e2e8c223c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f44c8ddecf42e5b36851b48164cf86"
I20260812 09:42:57.124545 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3cb7d3948c47acb806c1c332ef1c24" candidate_uuid: "5d973305767c40f5b9a261e2e8c223c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd797339c5142cb8f148bd40c4e7266"
I20260812 09:42:57.124630 20482 raft_consensus.cc:3037] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:57.124670 20314 raft_consensus.cc:3037] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:42:57.125954 20314 raft_consensus.cc:2454] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d973305767c40f5b9a261e2e8c223c8 in term 1.
I20260812 09:42:57.125954 20482 raft_consensus.cc:2454] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d973305767c40f5b9a261e2e8c223c8 in term 1.
I20260812 09:42:57.126237 20562 leader_election.cc:302] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d973305767c40f5b9a261e2e8c223c8, 7bd797339c5142cb8f148bd40c4e7266; no voters:
I20260812 09:42:57.126408 20712 raft_consensus.cc:2781] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:42:57.126632 20712 raft_consensus.cc:686] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 1 LEADER]: Becoming Leader. State: Replica: 5d973305767c40f5b9a261e2e8c223c8, State: Running, Role: LEADER
I20260812 09:42:57.126956 20712 consensus_queue.cc:237] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:57.130164 20150 catalog_manager.cc:5495] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "5d973305767c40f5b9a261e2e8c223c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } health_report { overall_health: UNKNOWN } } }
W20260812 09:42:57.139279 20540 tablet.cc:2369] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:42:57.186868 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.183504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.3.147.3:42273)
0812 09:42:57.183639 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:57.183643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:57.183656 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:42:57.183741 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:57.183743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:57.183803 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:57.183903 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:42:57.184128 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.184673 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.185487 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:57.185712 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:57.185792 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:57.186056 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:42:57.186507 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:42:57.186600 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:57.186659 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 09:42:57.186719 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:42:57.193176 20482 raft_consensus.cc:1261] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 1 FOLLOWER]: Refusing update from remote peer 5d973305767c40f5b9a261e2e8c223c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:42:57.193277 20314 raft_consensus.cc:1261] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Refusing update from remote peer 5d973305767c40f5b9a261e2e8c223c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:42:57.193950 20712 consensus_queue.cc:1035] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:42:57.194114 20716 consensus_queue.cc:1035] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:42:57.201889 20720 mvcc.cc:205] Tried to move back new op lower bound from 7317617775375716352 to 7317617775114661888. Current Snapshot: MvccSnapshot[applied={T|T < 7317617775375716352}]
I20260812 09:42:57.209200 20149 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:42:57.209374 20149 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:42:57.211720 20149 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Sending DeleteTablet for 3 replicas of tablet 3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:42:57.212536 20454 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3cb7d3948c47acb806c1c332ef1c24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:57 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:44580
I20260812 09:42:57.212536 20615 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3cb7d3948c47acb806c1c332ef1c24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:57 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:34810
I20260812 09:42:57.212953 20288 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3cb7d3948c47acb806c1c332ef1c24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:42:57 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:55388
I20260812 09:42:57.213115 20729 tablet_replica.cc:330] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: stopping tablet replica
I20260812 09:42:57.213377 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 20192
I20260812 09:42:57.213399 20730 tablet_replica.cc:330] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: stopping tablet replica
I20260812 09:42:57.213575 20729 raft_consensus.cc:2229] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:42:57.213676 20730 raft_consensus.cc:2229] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:42:57.214010 20729 raft_consensus.cc:2258] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:42:57.214032 20730 raft_consensus.cc:2258] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:42:57.216408 20730 ts_tablet_manager.cc:1905] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:42:57.216454 20729 ts_tablet_manager.cc:1905] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:42:57.219221 20729 ts_tablet_manager.cc:1918] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:42:57.219221 20730 ts_tablet_manager.cc:1918] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:42:57.219326 20730 log.cc:1196] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:42:57.219455 20729 log.cc:1196] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:42:57.219740 20730 ts_tablet_manager.cc:1939] T 3e3cb7d3948c47acb806c1c332ef1c24 P 5d973305767c40f5b9a261e2e8c223c8: Deleting consensus metadata
I20260812 09:42:57.219904 20729 ts_tablet_manager.cc:1939] T 3e3cb7d3948c47acb806c1c332ef1c24 P f6f44c8ddecf42e5b36851b48164cf86: Deleting consensus metadata
I20260812 09:42:57.220917 20128 catalog_manager.cc:4841] TS 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273): tablet 3e3cb7d3948c47acb806c1c332ef1c24 (table test-table [id=fa9057fcbc174a7da8840e6a38e533d7]) successfully deleted
I20260812 09:42:57.221254 20128 catalog_manager.cc:4841] TS f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993): tablet 3e3cb7d3948c47acb806c1c332ef1c24 (table test-table [id=fa9057fcbc174a7da8840e6a38e533d7]) successfully deleted
W20260812 09:42:57.229017 20130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:45893 (error 108)
I20260812 09:42:57.229422 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 20369
I20260812 09:42:57.231930 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.231589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:45893 (local address 127.0.0.1:55392)
0812 09:42:57.231773 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.231847 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260812 09:42:57.232085 20130 catalog_manager.cc:4584] TS 7bd797339c5142cb8f148bd40c4e7266 (127.3.147.1:45893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e3cb7d3948c47acb806c1c332ef1c24: Network error: Client connection negotiation failed: client connection to 127.3.147.1:45893: connect: Connection refused (error 111)
I20260812 09:42:57.243499 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 20543
I20260812 09:42:57.253046 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 20109
I20260812 09:42:57.261654 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44477
--webserver_interface=127.3.147.62
--webserver_port=39247
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:44477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:57.373521 20733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:57.373828 20733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:57.373910 20733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:57.377944 20733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:42:57.378031 20733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:57.378067 20733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:57.378134 20733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:42:57.378188 20733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:42:57.383523 20733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:44477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:44477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=39247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:57.384656 20733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:57.385717 20733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:57.394359 20738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:57.394428 20741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:57.394346 20739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:57.394836 20733 server_base.cc:1034] running on GCE node
I20260812 09:42:57.395426 20733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:57.396554 20733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:57.397749 20733 hybrid_clock.cc:648] HybridClock initialized: now 1786527777397720 us; error 53 us; skew 500 ppm
Aug 12 09:42:57 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527777, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:57.400583 20733 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:42:57.401439 20733 webserver.cc:460] Webserver started at http://127.3.147.62:39247/ using document root <none> and password file <none>
I20260812 09:42:57.401968 20733 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:57.402029 20733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:57.405776 20733 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:42:57.407812 20748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:57.409164 20733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:42:57.409271 20733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "51a141c10c9940fa975403010a5558bd"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "5bee367ff1418bc4763e55328789301b"
server_key_iv: "616a60d08f5d37220ccb59e20f8a7117"
server_key_version: "encryptionkey@0"
I20260812 09:42:57.409806 20733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:57.419764 20733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:57.422830 20733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:57.422974 20733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:57.430068 20733 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:44477
I20260812 09:42:57.430358 20809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:44477 every 8 connection(s)
I20260812 09:42:57.431134 20733 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:42:57.435608 20810 sys_catalog.cc:263] Verifying existing consensus state
I20260812 09:42:57.437191 20810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Bootstrap starting.
I20260812 09:42:57.437886 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 20733
I20260812 09:42:57.438338 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:45893
--local_ip_for_outbound_sockets=127.3.147.1
--tserver_master_addrs=127.3.147.62:44477
--webserver_port=46353
--webserver_interface=127.3.147.1
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:42:57.455302 20810 log.cc:824] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:57.466682 20810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:42:57.467060 20810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd: Bootstrap complete.
I20260812 09:42:57.471091 20810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:57.471520 20810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a141c10c9940fa975403010a5558bd, State: Initialized, Role: FOLLOWER
I20260812 09:42:57.472000 20810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:57.472146 20810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:42:57.472260 20810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:42:57.472388 20810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 2 FOLLOWER]: Advancing to term 3
I20260812 09:42:57.474216 20810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:57.474547 20810 leader_election.cc:302] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51a141c10c9940fa975403010a5558bd; no voters:
I20260812 09:42:57.474859 20810 leader_election.cc:288] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 09:42:57.474995 20815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 3 FOLLOWER]: Leader election won for term 3
I20260812 09:42:57.475244 20815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [term 3 LEADER]: Becoming Leader. State: Replica: 51a141c10c9940fa975403010a5558bd, State: Running, Role: LEADER
I20260812 09:42:57.475541 20815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } }
I20260812 09:42:57.476070 20810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:42:57.476995 20816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "51a141c10c9940fa975403010a5558bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } } }
I20260812 09:42:57.477144 20816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:57.477524 20817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51a141c10c9940fa975403010a5558bd. Latest consensus state: current_term: 3 leader_uuid: "51a141c10c9940fa975403010a5558bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a141c10c9940fa975403010a5558bd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 44477 } } }
I20260812 09:42:57.477672 20817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a141c10c9940fa975403010a5558bd [sys.catalog]: This master's current role is: LEADER
I20260812 09:42:57.478086 20825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:42:57.483143 20825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:42:57.483888 20825 catalog_manager.cc:1259] Loaded cluster ID: 93b5d2d89e9d433f9d758248f5dad2d5
I20260812 09:42:57.483954 20825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 09:42:57.485137 20825 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f35630f99ad google::LogMessage::Fail() at ??:0
@ 0x7f35630fc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f35630f94de google::LogMessage::Flush() at ??:0
@ 0x7f35630faa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f356da38fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f356da39d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f356da36579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f356da7a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f356534b3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f356534bc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f356534d3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f356533c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f35669806db start_thread at ??:0
@ 0x7f3561bf761f clone at ??:0
W20260812 09:42:57.581475 20812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:57.581713 20812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:57.581754 20812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:57.585788 20812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:57.585841 20812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:57.585897 20812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:42:57.591174 20812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:45893
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=46353
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:57.592325 20812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:57.593370 20812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:57.602988 20862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:57.602991 20859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:57.603492 20812 server_base.cc:1034] running on GCE node
W20260812 09:42:57.603554 20858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:57.603945 20812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:57.604668 20812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:57.605863 20812 hybrid_clock.cc:648] HybridClock initialized: now 1786527777605838 us; error 48 us; skew 500 ppm
Aug 12 09:42:57 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527777, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:57.609014 20812 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:42:57.610258 20812 webserver.cc:460] Webserver started at http://127.3.147.1:46353/ using document root <none> and password file <none>
I20260812 09:42:57.611209 20812 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:57.611322 20812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:57.617947 20812 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 09:42:57.621228 20869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:57.622957 20812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:42:57.623114 20812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "7bd797339c5142cb8f148bd40c4e7266"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "8eb53062b1f0e6a6eebc7daee0e93341"
server_key_iv: "36fcf6447f7c7859c7b9dda957f14720"
server_key_version: "encryptionkey@0"
I20260812 09:42:57.623941 20812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:57.650534 20812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:57.654848 20812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:57.655059 20812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:57.655802 20812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:57.657191 20878 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 09:42:57.661484 20812 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 09:42:57.661574 20812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 09:42:57.661665 20812 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 09:42:57.664111 20812 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 09:42:57.664196 20812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260812 09:42:57.664523 20878 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Bootstrap starting.
I20260812 09:42:57.675316 20812 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:45893
I20260812 09:42:57.675395 21000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:45893 every 8 connection(s)
I20260812 09:42:57.676394 20812 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:42:57.685653 20878 log.cc:824] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Log is configured to *not* fsync() on all Append() calls
I20260812 09:42:57.686326 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 20812
I20260812 09:42:57.686879 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:37993
--local_ip_for_outbound_sockets=127.3.147.2
--tserver_master_addrs=127.3.147.62:44477
--webserver_port=46855
--webserver_interface=127.3.147.2
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:42:57.693270 20878 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:42:57.693677 20878 tablet_bootstrap.cc:492] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Bootstrap complete.
I20260812 09:42:57.694906 20878 ts_tablet_manager.cc:1397] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.008s
I20260812 09:42:57.697320 20878 raft_consensus.cc:348] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:57.697710 20878 raft_consensus.cc:729] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd797339c5142cb8f148bd40c4e7266, State: Initialized, Role: FOLLOWER
I20260812 09:42:57.698166 20878 consensus_queue.cc:260] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:57.699083 20878 ts_tablet_manager.cc:1428] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
W20260812 09:42:57.790498 21004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:42:57.678533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:42201)
0812 09:42:57.679143 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.679212 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:57.680362 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:57.789447 (+109085us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":127,"threads_started":1}
W20260812 09:42:57.791102 21001 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:44477 (0 consecutive failures): Network error: Failed to ping master at 127.3.147.62:44477: Client connection negotiation failed: client connection to 127.3.147.62:44477: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 09:42:57.791883 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.791574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:60221)
0812 09:42:57.791735 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.791814 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 09:42:57.792634 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.792431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:43939)
0812 09:42:57.792534 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.792576 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 09:42:57.792876 21001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 09:42:57.804761 21006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:57.805017 21006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:57.805122 21006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:57.809063 21006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:57.809134 21006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:57.809253 21006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:42:57.814324 21006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:37993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=46855
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:57.815310 21006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:57.816370 21006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:57.826035 21013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:57.826025 21015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:57.826026 21012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:57.826308 21006 server_base.cc:1034] running on GCE node
I20260812 09:42:57.826826 21006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:57.827487 21006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:57.828706 21006 hybrid_clock.cc:648] HybridClock initialized: now 1786527777828684 us; error 38 us; skew 500 ppm
Aug 12 09:42:57 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527777, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:57.831521 21006 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:42:57.832432 21006 webserver.cc:460] Webserver started at http://127.3.147.2:46855/ using document root <none> and password file <none>
I20260812 09:42:57.833011 21006 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:57.833106 21006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:57.837061 21006 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:42:57.839143 21024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:57.840425 21006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:42:57.840555 21006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "f6f44c8ddecf42e5b36851b48164cf86"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "ec4b0a345cf2e1a9e3361042be5293f5"
server_key_iv: "69db21883d312525db09674e5a8d4921"
server_key_version: "encryptionkey@0"
I20260812 09:42:57.841102 21006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:57.865947 21006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:57.868763 21006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:57.868971 21006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:57.869568 21006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:57.870414 21006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:57.870488 21006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:57.870563 21006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:57.870608 21006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:57.879976 21006 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:37993
I20260812 09:42:57.880004 21149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:37993 every 8 connection(s)
I20260812 09:42:57.881011 21006 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:42:57.884639 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.883109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:44829)
0812 09:42:57.883782 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.883889 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":112,"threads_started":1}
W20260812 09:42:57.885149 21150 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:44477 (0 consecutive failures): Network error: Failed to ping master at 127.3.147.62:44477: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
I20260812 09:42:57.885708 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.885498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:56283)
0812 09:42:57.885605 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.885650 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 09:42:57.886340 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:57.886123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:55463)
0812 09:42:57.886244 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:57.886296 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 09:42:57.886507 21150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 09:42:57.886898 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 21006
I20260812 09:42:57.887293 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:42273
--local_ip_for_outbound_sockets=127.3.147.3
--tserver_master_addrs=127.3.147.62:44477
--webserver_port=43955
--webserver_interface=127.3.147.3
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:42:58.002326 21153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:42:58.002580 21153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:42:58.002626 21153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:42:58.006620 21153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:42:58.006676 21153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:42:58.006735 21153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:42:58.011814 21153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:42273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=43955
--tserver_master_addrs=127.3.147.62:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:42:58.012876 21153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:42:58.013878 21153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:42:58.023588 21159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:58.023651 21162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:42:58.023651 21160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:42:58.023867 21153 server_base.cc:1034] running on GCE node
I20260812 09:42:58.024447 21153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:42:58.025053 21153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:42:58.026237 21153 hybrid_clock.cc:648] HybridClock initialized: now 1786527778026208 us; error 41 us; skew 500 ppm
Aug 12 09:42:58 dist-test-slave-cbbz krb5kdc[19481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527778, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:42:58.029034 21153 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:42:58.029901 21153 webserver.cc:460] Webserver started at http://127.3.147.3:43955/ using document root <none> and password file <none>
I20260812 09:42:58.030386 21153 fs_manager.cc:359] Metadata directory not provided
I20260812 09:42:58.030462 21153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:42:58.034297 21153 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:42:58.036612 21171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:58.037771 21153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:42:58.037865 21153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "5d973305767c40f5b9a261e2e8c223c8"
format_stamp: "Formatted at 2026-08-12 09:42:55 on dist-test-slave-cbbz"
server_key: "1915d992dd5df268ac653d8381f8624c"
server_key_iv: "bbec778fa0d946c6e90a7a019d5318a1"
server_key_version: "encryptionkey@0"
I20260812 09:42:58.038375 21153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:42:58.057564 21153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:42:58.060890 21153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:42:58.061041 21153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:42:58.061564 21153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:42:58.062353 21153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:42:58.062399 21153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:58.062438 21153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:42:58.062458 21153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:42:58.071605 21153 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42273
I20260812 09:42:58.071638 21300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42273 every 8 connection(s)
I20260812 09:42:58.072603 21153 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:42:58.073773 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 21153
I20260812 09:42:58.076357 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:58.074761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:41963)
0812 09:42:58.075456 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:58.075555 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":148,"threads_started":1}
W20260812 09:42:58.076777 21301 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:44477 (0 consecutive failures): Network error: Failed to ping master at 127.3.147.62:44477: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
I20260812 09:42:58.077373 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:58.077144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:35223)
0812 09:42:58.077264 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:58.077319 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 09:42:58.078030 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:58.077825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:39289)
0812 09:42:58.077928 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:58.077974 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 09:42:58.078228 21301 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 09:42:58.793886 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:58.793465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:36661)
0812 09:42:58.793712 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:58.793765 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":77,"threads_started":1}
I20260812 09:42:58.887447 21308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:58.887047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:58221)
0812 09:42:58.887292 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:58.887346 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260812 09:42:59.079224 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:59.078792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:51031)
0812 09:42:59.079048 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:59.079112 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260812 09:42:59.350962 21310 raft_consensus.cc:482] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:42:59.351243 21310 raft_consensus.cc:504] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:42:59.352452 21310 leader_election.cc:288] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993), 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273)
Aug 12 09:42:59 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527777, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.2@KRBTEST.COM
Aug 12 09:42:59 dist-test-slave-cbbz krb5kdc[19481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527777, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.3@KRBTEST.COM
I20260812 09:42:59.366986 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:59.352705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:37993 (local address 127.3.147.1:38215)
0812 09:42:59.352926 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:59.352940 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:59.353046 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:59.354829 (+ 1783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:59.354836 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:59.355388 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:59.355809 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:59.355826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:59.357230 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:59.357232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:59.358240 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:59.358251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:59.358984 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:59.358988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:59.359224 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:59.359995 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:59.360007 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:59.361849 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:59.364138 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:59.364177 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:59.364191 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:59.364534 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:59.364967 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:59.364970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:59.364971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:59.365171 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:59.365899 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:59.365905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:59.366193 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 09:42:59.366377 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 09:42:59.366778 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":220,"spinlock_wait_cycles":9344,"thread_start_us":80,"threads_started":1}
I20260812 09:42:59.366984 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:59.353383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:60505 (local address 127.3.147.3:42273)
0812 09:42:59.353740 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:59.353760 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:59.353793 (+ 33us) server_negotiation.cc:407] Connection header received
0812 09:42:59.354429 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:59.354435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:59.354803 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:59.355099 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:59.355997 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:59.357052 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:59.358374 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:59.358849 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:59.359024 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:59.362477 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:59.362483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:59.362494 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:59.362522 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:59.364568 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:59.365087 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:59.365093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:59.365098 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:59.365232 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:59.365521 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:59.365524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:59.365525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:59.365957 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:59.366085 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:59.366493 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 09:42:59.366775 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":149,"threads_started":1}
I20260812 09:42:59.366988 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:59.352842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:38215 (local address 127.3.147.2:37993)
0812 09:42:59.353141 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 09:42:59.353175 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0812 09:42:59.353215 (+ 40us) server_negotiation.cc:407] Connection header received
0812 09:42:59.354064 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:42:59.354079 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:42:59.354440 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:42:59.354701 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:42:59.356000 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:59.357053 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:59.358372 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:42:59.358851 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:42:59.359030 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:59.362094 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:42:59.362101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:42:59.362117 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:42:59.362153 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:42:59.364010 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:59.364673 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:59.364680 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:59.364686 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:59.364859 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:42:59.365274 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:42:59.365294 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:42:59.365299 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:42:59.365775 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:42:59.365880 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 09:42:59.366491 (+ 611us) server_negotiation.cc:299] Negotiation successful
0812 09:42:59.366790 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":101,"threads_started":1}
I20260812 09:42:59.367043 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:59.353066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42273 (local address 127.3.147.1:60505)
0812 09:42:59.353385 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:59.353396 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 09:42:59.353479 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:42:59.355067 (+ 1588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:42:59.355072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:42:59.355340 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:42:59.355808 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:59.355825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:59.357239 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:59.357242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:59.358238 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:42:59.358250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:42:59.358986 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:42:59.358990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:42:59.359224 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:42:59.359959 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:42:59.359987 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:42:59.362263 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:42:59.364676 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:59.364686 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:59.364688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:59.364979 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:59.365327 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:42:59.365332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:42:59.365337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:42:59.365425 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:42:59.366068 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:42:59.366071 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:42:59.366180 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 09:42:59.366378 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 09:42:59.366866 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":325,"spinlock_wait_cycles":8064,"thread_start_us":54,"threads_started":1}
W20260812 09:42:59.370169 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
W20260812 09:42:59.370355 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:42:59.370401 20870 leader_election.cc:302] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bd797339c5142cb8f148bd40c4e7266; no voters: 5d973305767c40f5b9a261e2e8c223c8, f6f44c8ddecf42e5b36851b48164cf86
I20260812 09:42:59.370636 21310 raft_consensus.cc:2726] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:42:59.795056 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:59.794718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:40315)
0812 09:42:59.794892 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:59.794962 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 09:42:59.888617 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:42:59.888220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:33565)
0812 09:42:59.888470 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 09:42:59.888527 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":87,"threads_started":1}
I20260812 09:43:00.080379 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:00.079908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:57677)
0812 09:43:00.080233 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:00.080290 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260812 09:43:00.796391 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:00.795866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:53129)
0812 09:43:00.796152 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:00.796275 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
I20260812 09:43:00.889765 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:00.889324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:37873)
0812 09:43:00.889585 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:00.889638 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260812 09:43:01.081540 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:01.081110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:47729)
0812 09:43:01.081381 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:01.081433 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
I20260812 09:43:01.391570 21322 raft_consensus.cc:482] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:01.391703 21322 raft_consensus.cc:504] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:43:01.392227 21322 leader_election.cc:288] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993), 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273)
W20260812 09:43:01.393237 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
W20260812 09:43:01.393409 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:43:01.393465 20870 leader_election.cc:302] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bd797339c5142cb8f148bd40c4e7266; no voters: 5d973305767c40f5b9a261e2e8c223c8, f6f44c8ddecf42e5b36851b48164cf86
I20260812 09:43:01.393666 21322 raft_consensus.cc:2726] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:43:01.797593 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:01.797145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:60095)
0812 09:43:01.797426 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:01.797478 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":78,"threads_started":1}
I20260812 09:43:01.890846 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:01.890458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:51761)
0812 09:43:01.890708 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:01.890759 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
I20260812 09:43:02.082695 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:02.082235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:57455)
0812 09:43:02.082531 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:02.082594 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
I20260812 09:43:02.799114 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:02.798593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:48377)
0812 09:43:02.798939 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:02.799004 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":87,"threads_started":1}
I20260812 09:43:02.891996 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:02.891611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:46349)
0812 09:43:02.891855 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:02.891909 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260812 09:43:03.084098 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:03.083645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:35591)
0812 09:43:03.083940 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:03.083996 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
I20260812 09:43:03.800480 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:03.799981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:52271)
0812 09:43:03.800319 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:03.800378 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":94,"threads_started":1}
I20260812 09:43:03.893352 21330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:03.892836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:55967)
0812 09:43:03.893116 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:03.893179 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":78,"threads_started":1}
I20260812 09:43:04.085345 21332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:04.084897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:41613)
0812 09:43:04.085185 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:04.085243 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
I20260812 09:43:04.358570 21333 raft_consensus.cc:482] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:04.358707 21333 raft_consensus.cc:504] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:43:04.359232 21333 leader_election.cc:288] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993), 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273)
W20260812 09:43:04.360060 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
W20260812 09:43:04.360245 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:43:04.360294 20870 leader_election.cc:302] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bd797339c5142cb8f148bd40c4e7266; no voters: 5d973305767c40f5b9a261e2e8c223c8, f6f44c8ddecf42e5b36851b48164cf86
I20260812 09:43:04.360435 21333 raft_consensus.cc:2726] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:43:04.802166 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:04.801513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:36473)
0812 09:43:04.801874 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:04.801972 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":130,"threads_started":1}
I20260812 09:43:04.894650 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:04.894139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:44415)
0812 09:43:04.894452 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:04.894519 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":75,"threads_started":1}
I20260812 09:43:05.086514 21336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:05.086047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:36039)
0812 09:43:05.086361 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:05.086416 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":98,"threads_started":1}
I20260812 09:43:05.803480 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:05.803007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:37905)
0812 09:43:05.803317 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:05.803377 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":100,"threads_started":1}
I20260812 09:43:05.895820 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:05.895413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:38729)
0812 09:43:05.895673 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:05.895726 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260812 09:43:06.087874 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:06.087331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:37375)
0812 09:43:06.087677 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:06.087756 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":115,"threads_started":1}
I20260812 09:43:06.804760 21342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:06.804304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:48107)
0812 09:43:06.804595 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:06.804650 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":91,"threads_started":1}
I20260812 09:43:06.897008 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:06.896570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:58235)
0812 09:43:06.896857 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:06.896913 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":93,"threads_started":1}
I20260812 09:43:07.089006 21344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:07.088567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:54373)
0812 09:43:07.088853 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:07.088910 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":99,"threads_started":1}
I20260812 09:43:07.806205 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:07.805739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:53701)
0812 09:43:07.806031 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:07.806095 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":81,"threads_started":1}
I20260812 09:43:07.898226 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:07.897795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:48037)
0812 09:43:07.898064 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:07.898123 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":79,"threads_started":1}
I20260812 09:43:08.090248 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:08.089819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:42819)
0812 09:43:08.090090 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:08.090148 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":98,"threads_started":1}
I20260812 09:43:08.150916 21348 raft_consensus.cc:482] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:08.151068 21348 raft_consensus.cc:504] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:43:08.151552 21348 leader_election.cc:288] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993), 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273)
W20260812 09:43:08.152484 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
W20260812 09:43:08.152650 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:43:08.152716 20870 leader_election.cc:302] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bd797339c5142cb8f148bd40c4e7266; no voters: 5d973305767c40f5b9a261e2e8c223c8, f6f44c8ddecf42e5b36851b48164cf86
I20260812 09:43:08.152905 21348 raft_consensus.cc:2726] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:43:08.807619 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:08.807151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:54145)
0812 09:43:08.807455 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:08.807517 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":84,"threads_started":1}
I20260812 09:43:08.899466 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:08.899007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:51487)
0812 09:43:08.899307 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:08.899367 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":97,"threads_started":1}
I20260812 09:43:09.091377 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:09.090974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:55745)
0812 09:43:09.091231 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:09.091284 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260812 09:43:09.808912 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:09.808470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:45773)
0812 09:43:09.808751 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:09.808810 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":80,"threads_started":1}
I20260812 09:43:09.900710 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:09.900278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:55163)
0812 09:43:09.900552 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:09.900610 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260812 09:43:10.092532 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:10.092083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:39137)
0812 09:43:10.092374 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:10.092428 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":130,"threads_started":1}
I20260812 09:43:10.421290 21355 raft_consensus.cc:482] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:10.421422 21355 raft_consensus.cc:504] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:43:10.421772 21355 leader_election.cc:288] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993), 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273)
W20260812 09:43:10.422580 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
W20260812 09:43:10.422763 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:43:10.422832 20870 leader_election.cc:302] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bd797339c5142cb8f148bd40c4e7266; no voters: 5d973305767c40f5b9a261e2e8c223c8, f6f44c8ddecf42e5b36851b48164cf86
I20260812 09:43:10.422987 21355 raft_consensus.cc:2726] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:43:10.810178 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:10.809754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:34641)
0812 09:43:10.810017 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:10.810071 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260812 09:43:10.902036 21357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:10.901586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:44867)
0812 09:43:10.901881 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:10.901941 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":108,"threads_started":1}
I20260812 09:43:11.093689 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:11.093268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:57185)
0812 09:43:11.093535 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:11.093591 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":102,"threads_started":1}
I20260812 09:43:11.811487 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:11.811030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:36645)
0812 09:43:11.811336 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:11.811389 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":108,"threads_started":1}
I20260812 09:43:11.903249 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:11.902770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:49383)
0812 09:43:11.903079 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:11.903144 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":82,"threads_started":1}
I20260812 09:43:12.094900 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:12.094443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:48297)
0812 09:43:12.094748 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:12.094802 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":133,"threads_started":1}
I20260812 09:43:12.408812 21363 raft_consensus.cc:482] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:12.408949 21363 raft_consensus.cc:504] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f44c8ddecf42e5b36851b48164cf86" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 37993 } } peers { permanent_uuid: "5d973305767c40f5b9a261e2e8c223c8" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42273 } } peers { permanent_uuid: "7bd797339c5142cb8f148bd40c4e7266" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 45893 } }
I20260812 09:43:12.409324 21363 leader_election.cc:288] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993), 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273)
W20260812 09:43:12.410228 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6f44c8ddecf42e5b36851b48164cf86 (127.3.147.2:37993): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
W20260812 09:43:12.410382 20870 leader_election.cc:341] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d973305767c40f5b9a261e2e8c223c8 (127.3.147.3:42273): Not found: Tablet not found: 3e3cb7d3948c47acb806c1c332ef1c24
I20260812 09:43:12.410449 20870 leader_election.cc:302] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bd797339c5142cb8f148bd40c4e7266; no voters: 5d973305767c40f5b9a261e2e8c223c8, f6f44c8ddecf42e5b36851b48164cf86
I20260812 09:43:12.410665 21363 raft_consensus.cc:2726] T 3e3cb7d3948c47acb806c1c332ef1c24 P 7bd797339c5142cb8f148bd40c4e7266 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:43:12.813279 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:12.812730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.1:54409)
0812 09:43:12.813077 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:12.813149 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":136,"threads_started":1}
I20260812 09:43:12.904662 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:12.904246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.2:39141)
0812 09:43:12.904491 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:12.904546 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260812 09:43:13.082959 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 20812
I20260812 09:43:13.091630 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 21006
I20260812 09:43:13.097838 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:13.096355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:44477 (local address 127.3.147.3:56153)
0812 09:43:13.097689 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:13.097744 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:44477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1210,"thread_start_us":71,"threads_started":1}
I20260812 09:43:13.102397 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 21153
2026-08-12T09:43:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19436 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:43:13 dist-test-slave-cbbz krb5kdc[21370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:43:13 dist-test-slave-cbbz krb5kdc[21370](info): set up 2 sockets
Aug 12 09:43:13 dist-test-slave-cbbz krb5kdc[21370](info): commencing operation
krb5kdc: starting...
W20260812 09:43:14.225209 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:43:14 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:43:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:43:14.348979 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38965
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:38965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:43:14.458920 21386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:14.459183 21386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:14.459223 21386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:14.463116 21386 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 09:43:14.463162 21386 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 09:43:14.463178 21386 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 09:43:14.463192 21386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:43:14.463209 21386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:14.463227 21386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:14.463248 21386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:43:14.463267 21386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:43:14.468314 21386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:38965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:14.469229 21386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:14.470144 21386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:14.478685 21395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:14.478686 21391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:14.478790 21386 server_base.cc:1034] running on GCE node
W20260812 09:43:14.478745 21392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:14.479522 21386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:14.480687 21386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:14.481874 21386 hybrid_clock.cc:648] HybridClock initialized: now 1786527794481841 us; error 47 us; skew 500 ppm
Aug 12 09:43:14 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527794, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:14.484664 21386 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:43:14.485535 21386 webserver.cc:460] Webserver started at http://127.3.147.62:37671/ using document root <none> and password file <none>
I20260812 09:43:14.486079 21386 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:14.486177 21386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:14.486406 21386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:43:14.488466 21386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "e35011e296454af99c25eba73ae2103b"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "6508ba3ecf5162d3f591099791cd9887"
server_key_iv: "b440fea6347ec89523eb3a37a9438881"
server_key_version: "encryptionkey@0"
I20260812 09:43:14.489032 21386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "e35011e296454af99c25eba73ae2103b"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "6508ba3ecf5162d3f591099791cd9887"
server_key_iv: "b440fea6347ec89523eb3a37a9438881"
server_key_version: "encryptionkey@0"
I20260812 09:43:14.492627 21386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:43:14.495029 21402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:14.496320 21386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:43:14.496459 21386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "e35011e296454af99c25eba73ae2103b"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "6508ba3ecf5162d3f591099791cd9887"
server_key_iv: "b440fea6347ec89523eb3a37a9438881"
server_key_version: "encryptionkey@0"
I20260812 09:43:14.496567 21386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:14.515285 21386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:14.518558 21386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:14.518769 21386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:14.526058 21386 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:38965
I20260812 09:43:14.526363 21463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:38965 every 8 connection(s)
I20260812 09:43:14.527016 21386 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:43:14.529816 21464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:43:14.535018 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 21386
I20260812 09:43:14.535197 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:43:14.535543 3660 external_mini_cluster.cc:1428] Setting key 4f229014e57b48f9dfbb23bdbbe7b2ad
I20260812 09:43:14.535521 21464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Bootstrap starting.
I20260812 09:43:14.538271 21464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Neither blocks nor log segments found. Creating new log.
I20260812 09:43:14.539163 21464 log.cc:824] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:14.541276 21464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: No bootstrap required, opened a new log
Aug 12 09:43:14 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:43:14.545642 21464 raft_consensus.cc:348] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:14.545881 21464 raft_consensus.cc:374] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:43:14.546005 21464 raft_consensus.cc:729] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35011e296454af99c25eba73ae2103b, State: Initialized, Role: FOLLOWER
I20260812 09:43:14.546525 21464 consensus_queue.cc:260] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:14.546720 21464 raft_consensus.cc:388] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:43:14.546790 21464 raft_consensus.cc:482] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:43:14.546895 21464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:43:14.548025 21464 raft_consensus.cc:504] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:14.548389 21464 leader_election.cc:302] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e35011e296454af99c25eba73ae2103b; no voters:
I20260812 09:43:14.548754 21464 leader_election.cc:288] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:43:14.548894 21470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:43:14.549134 21470 raft_consensus.cc:686] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 1 LEADER]: Becoming Leader. State: Replica: e35011e296454af99c25eba73ae2103b, State: Running, Role: LEADER
I20260812 09:43:14.549464 21470 consensus_queue.cc:237] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:14.550050 21464 sys_catalog.cc:564] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:43:14.551105 21471 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e35011e296454af99c25eba73ae2103b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } } }
I20260812 09:43:14.551196 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:14.537017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.3.147.62:38965)
0812 09:43:14.537463 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:14.537479 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:14.537514 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:43:14.538118 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:14.538133 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:14.538499 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:14.538773 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:14.539716 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:14.540828 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:14.541977 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:14.542437 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:14.542569 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:14.545389 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:14.545395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:14.545408 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:14.545434 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:14.547512 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:14.547924 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:14.547933 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:14.547941 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:14.548167 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:14.548461 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:14.548464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:14.548466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:14.548971 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:14.549737 (+ 766us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:14.549889 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 09:43:14.550190 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":159,"threads_started":1}
I20260812 09:43:14.551230 21471 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: This master's current role is: LEADER
I20260812 09:43:14.551563 21472 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35011e296454af99c25eba73ae2103b. Latest consensus state: current_term: 1 leader_uuid: "e35011e296454af99c25eba73ae2103b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } } }
I20260812 09:43:14.551642 21472 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: This master's current role is: LEADER
W20260812 09:43:14.555411 21488 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:43:14.555495 21488 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:43:14.555552 21482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:43:14.556463 21482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:43:14.562157 21482 catalog_manager.cc:1347] Generated new cluster ID: 660e6563a012411eb50796cc9b8ec33a
I20260812 09:43:14.562233 21482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:43:14.578823 21482 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:43:14.579805 21482 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:43:14.594143 21482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Generated new TSK 0
I20260812 09:43:14.594898 21482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:43:14.628835 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38965
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:43:14.745515 21495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:14.745769 21495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:14.745811 21495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:14.749778 21495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:14.749828 21495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:14.749883 21495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:43:14.755116 21495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:14.756084 21495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:14.757081 21495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:14.767032 21504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:14.767037 21500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:14.767112 21495 server_base.cc:1034] running on GCE node
W20260812 09:43:14.767058 21501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:14.767700 21495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:14.768443 21495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:14.769636 21495 hybrid_clock.cc:648] HybridClock initialized: now 1786527794769623 us; error 46 us; skew 500 ppm
Aug 12 09:43:14 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527794, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:14.772559 21495 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:43:14.773422 21495 webserver.cc:460] Webserver started at http://127.3.147.1:33233/ using document root <none> and password file <none>
I20260812 09:43:14.773988 21495 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:14.774082 21495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:14.774315 21495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:43:14.776404 21495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "e7e98c354fe84854a31b3e7a00408844"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "639c9edb9e48c4b4c8da861153ffd740"
server_key_iv: "295ed69c2bb43cc2b5706102438e41c9"
server_key_version: "encryptionkey@0"
I20260812 09:43:14.777002 21495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "e7e98c354fe84854a31b3e7a00408844"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "639c9edb9e48c4b4c8da861153ffd740"
server_key_iv: "295ed69c2bb43cc2b5706102438e41c9"
server_key_version: "encryptionkey@0"
I20260812 09:43:14.780682 21495 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:43:14.783078 21511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:14.784369 21495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:43:14.784502 21495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "e7e98c354fe84854a31b3e7a00408844"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "639c9edb9e48c4b4c8da861153ffd740"
server_key_iv: "295ed69c2bb43cc2b5706102438e41c9"
server_key_version: "encryptionkey@0"
I20260812 09:43:14.784612 21495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:14.800275 21495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:14.803085 21495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:14.803288 21495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:14.803879 21495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:14.804744 21495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:43:14.804813 21495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:14.804879 21495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:43:14.804932 21495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:14.814739 21495 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:46269
I20260812 09:43:14.814788 21640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:46269 every 8 connection(s)
I20260812 09:43:14.815604 21495 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
Aug 12 09:43:14 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527794, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:43:14.825702 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 21495
I20260812 09:43:14.825845 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:43:14.826201 3660 external_mini_cluster.cc:1428] Setting key 49b6b4f1b462ee9ee2f0ac3b79d5fd6a
I20260812 09:43:14.829391 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:14.817384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:55605 (local address 127.3.147.62:38965)
0812 09:43:14.817518 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:14.817522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:14.818808 (+ 1286us) server_negotiation.cc:407] Connection header received
0812 09:43:14.819758 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:14.819762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:14.819822 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:14.819898 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:14.821135 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:14.821683 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:14.822831 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:14.823080 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:14.823202 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:14.825593 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:14.825603 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:14.825606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:14.825633 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:14.827257 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:14.827783 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:14.827789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:14.827790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:14.827882 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:14.828257 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:14.828262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:14.828264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:14.828452 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:14.828561 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:14.829070 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 09:43:14.829223 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 09:43:14.830324 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:14.817695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:55605)
0812 09:43:14.818642 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:14.818694 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:14.819534 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:14.820062 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:14.820069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:14.820498 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:14.820980 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:14.820994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:14.821826 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:14.821830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:14.822699 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:14.822708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:14.823246 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:14.823251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:14.823422 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:14.823923 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:14.823939 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:14.825420 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:14.827384 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:14.827390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:14.827405 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:14.827674 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:14.827976 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:14.827980 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:14.827982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:14.828126 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:14.828561 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:14.828568 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:14.828822 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 09:43:14.829009 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 09:43:14.829343 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":623,"threads_started":1}
I20260812 09:43:14.831615 21641 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38965
I20260812 09:43:14.831871 21641 heartbeater.cc:464] Registering TS with master...
I20260812 09:43:14.832492 21641 heartbeater.cc:511] Master 127.3.147.62:38965 requested a full tablet report, sending...
I20260812 09:43:14.834342 21421 ts_manager.cc:194] Registered new tserver with Master: e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269)
I20260812 09:43:14.835997 21421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:55605
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:43:14.855594 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38965
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:43:14.967049 21646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:14.967289 21646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:14.967330 21646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:14.971284 21646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:14.971339 21646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:14.971441 21646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:43:14.976691 21646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:14.977717 21646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:14.978761 21646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:14.988296 21653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:14.988297 21656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:14.988327 21651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:14.988710 21646 server_base.cc:1034] running on GCE node
I20260812 09:43:14.989143 21646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:14.989753 21646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:14.990970 21646 hybrid_clock.cc:648] HybridClock initialized: now 1786527794990962 us; error 29 us; skew 500 ppm
Aug 12 09:43:14 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527794, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:14.993803 21646 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:43:14.994699 21646 webserver.cc:460] Webserver started at http://127.3.147.2:46141/ using document root <none> and password file <none>
I20260812 09:43:14.995337 21646 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:14.995436 21646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:14.995643 21646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:43:14.997717 21646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "36d95759bf4a4641a2c745da8567abe6"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "8995e397b4b9773811ed5d7b1916f86a"
server_key_iv: "9a4f20be0dbb19ae0126b447ea2913fa"
server_key_version: "encryptionkey@0"
I20260812 09:43:14.998287 21646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "36d95759bf4a4641a2c745da8567abe6"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "8995e397b4b9773811ed5d7b1916f86a"
server_key_iv: "9a4f20be0dbb19ae0126b447ea2913fa"
server_key_version: "encryptionkey@0"
I20260812 09:43:15.002239 21646 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:43:15.004598 21663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:15.005749 21646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:43:15.005884 21646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "36d95759bf4a4641a2c745da8567abe6"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "8995e397b4b9773811ed5d7b1916f86a"
server_key_iv: "9a4f20be0dbb19ae0126b447ea2913fa"
server_key_version: "encryptionkey@0"
I20260812 09:43:15.005996 21646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:15.025205 21646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:15.028479 21646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:15.028700 21646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:15.029287 21646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:15.030135 21646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:43:15.030203 21646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:15.030288 21646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:43:15.030336 21646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:15.039789 21646 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46085
I20260812 09:43:15.039821 21793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46085 every 8 connection(s)
I20260812 09:43:15.040696 21646 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:43:15.042296 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 21646
I20260812 09:43:15.042436 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:43:15.042793 3660 external_mini_cluster.cc:1428] Setting key a3bfc9bd9e935d123bc77751333cd240
Aug 12 09:43:15 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527794, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:43:15.056761 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.042510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:46717 (local address 127.3.147.62:38965)
0812 09:43:15.042680 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.042686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.043545 (+ 859us) server_negotiation.cc:407] Connection header received
0812 09:43:15.045290 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.045292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.045334 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.045403 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:15.046966 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.047850 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.049648 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.050028 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.050145 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.053069 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:15.053073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:15.053075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:15.053106 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:15.054801 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:15.055288 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:15.055291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:15.055293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:15.055369 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:15.055690 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:15.055695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:15.055701 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:15.055881 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:15.055959 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.056491 (+ 532us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.056596 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:43:15.057677 21797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.042911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:46717)
0812 09:43:15.043376 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.043449 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:15.045068 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:15.045583 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:15.045594 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:15.046132 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:15.046821 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.046837 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.047992 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.047998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.049508 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.049524 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.050215 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.050220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.050432 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.051140 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:15.051188 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:15.052888 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:15.054916 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:15.054920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:15.054931 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:15.055181 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:15.055428 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:15.055430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:15.055431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:15.055545 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:15.055981 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:15.055986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:15.056268 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 09:43:15.056424 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 09:43:15.056712 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":117,"threads_started":1}
I20260812 09:43:15.058846 21794 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38965
I20260812 09:43:15.059136 21794 heartbeater.cc:464] Registering TS with master...
I20260812 09:43:15.059646 21794 heartbeater.cc:511] Master 127.3.147.62:38965 requested a full tablet report, sending...
I20260812 09:43:15.060756 21421 ts_manager.cc:194] Registered new tserver with Master: 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085)
I20260812 09:43:15.061353 21421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:46717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:43:15.074790 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38965
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:43:15.187769 21802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:15.188122 21802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:15.188236 21802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:15.192288 21802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:15.192368 21802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:15.192454 21802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:43:15.197587 21802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:15.198566 21802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:15.199551 21802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:15.208829 21807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:15.208848 21811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:15.209076 21802 server_base.cc:1034] running on GCE node
W20260812 09:43:15.209156 21808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:15.209570 21802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:15.210275 21802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:15.211436 21802 hybrid_clock.cc:648] HybridClock initialized: now 1786527795211410 us; error 51 us; skew 500 ppm
Aug 12 09:43:15 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:15.214200 21802 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:43:15.215013 21802 webserver.cc:460] Webserver started at http://127.3.147.3:45151/ using document root <none> and password file <none>
I20260812 09:43:15.215556 21802 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:15.215617 21802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:15.215847 21802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:43:15.217828 21802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "09c989ce412a497c90399a4da816b383"
format_stamp: "Formatted at 2026-08-12 09:43:15 on dist-test-slave-cbbz"
server_key: "b3fa0d649b5ccf90746f787afeda80a4"
server_key_iv: "852079846513fb407a96bbbcec1377c6"
server_key_version: "encryptionkey@0"
I20260812 09:43:15.218370 21802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "09c989ce412a497c90399a4da816b383"
format_stamp: "Formatted at 2026-08-12 09:43:15 on dist-test-slave-cbbz"
server_key: "b3fa0d649b5ccf90746f787afeda80a4"
server_key_iv: "852079846513fb407a96bbbcec1377c6"
server_key_version: "encryptionkey@0"
I20260812 09:43:15.222019 21802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 09:43:15.224339 21820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:15.225481 21802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:43:15.225625 21802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "09c989ce412a497c90399a4da816b383"
format_stamp: "Formatted at 2026-08-12 09:43:15 on dist-test-slave-cbbz"
server_key: "b3fa0d649b5ccf90746f787afeda80a4"
server_key_iv: "852079846513fb407a96bbbcec1377c6"
server_key_version: "encryptionkey@0"
I20260812 09:43:15.225733 21802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:15.242961 21802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:15.245764 21802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:15.245970 21802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:15.246557 21802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:15.247428 21802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:43:15.247498 21802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:15.247565 21802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:43:15.247614 21802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:15.256836 21802 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:41343
I20260812 09:43:15.256884 21949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:41343 every 8 connection(s)
I20260812 09:43:15.257707 21802 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:43:15.261735 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 21802
I20260812 09:43:15.261888 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:43:15.262288 3660 external_mini_cluster.cc:1428] Setting key 99d0274eb176e5ba5e455250d4f0aa8e
Aug 12 09:43:15 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:43:15.271555 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.259555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:59737 (local address 127.3.147.62:38965)
0812 09:43:15.259705 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.259709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.260588 (+ 879us) server_negotiation.cc:407] Connection header received
0812 09:43:15.262156 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.262159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.262218 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.262285 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:15.263577 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.264409 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.265605 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.265813 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.265896 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.268196 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:15.268201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:15.268206 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:15.268236 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:15.269756 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:15.270241 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:15.270244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:15.270245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:15.270318 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:15.270607 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:15.270609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:15.270610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:15.270768 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:15.270843 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.271325 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.271439 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 09:43:15.272320 21952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.259899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:59737)
0812 09:43:15.260441 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.260482 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:15.261907 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:15.262504 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:15.262512 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:15.262892 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:15.263415 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.263434 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.264561 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.264563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.265488 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.265497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.265947 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.265950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.266082 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.266588 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:15.266604 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:15.267996 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:15.269867 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:15.269872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:15.269882 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:15.270130 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:15.270399 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:15.270401 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:15.270403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:15.270528 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:15.270857 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:15.270862 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:15.271068 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 09:43:15.271230 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 09:43:15.271461 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":229,"threads_started":1}
I20260812 09:43:15.273411 21950 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38965
I20260812 09:43:15.273607 21950 heartbeater.cc:464] Registering TS with master...
I20260812 09:43:15.274032 21950 heartbeater.cc:511] Master 127.3.147.62:38965 requested a full tablet report, sending...
I20260812 09:43:15.275005 21421 ts_manager.cc:194] Registered new tserver with Master: 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343)
I20260812 09:43:15.275738 21421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:59737
I20260812 09:43:15.276443 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:43:15.285264 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.278450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56740 (local address 127.3.147.62:38965)
0812 09:43:15.278597 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.278601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.278815 (+ 214us) server_negotiation.cc:407] Connection header received
0812 09:43:15.278891 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.278893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.278928 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.278987 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:15.279605 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.280077 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.280917 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.281116 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.281194 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.282476 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:15.282479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:15.282481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:15.282497 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:15.283761 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:15.284129 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:15.284132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:15.284133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:15.284256 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:15.284497 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:15.284499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:15.284500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:15.284701 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:15.284806 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.284994 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.285083 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:43:15.287843 21421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:43:15.291006 21421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:43:15.305711 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.299164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:46269 (local address 127.0.0.1:46358)
0812 09:43:15.299505 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.299554 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:15.299690 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:15.300901 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:15.300908 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:15.300941 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:15.301085 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.301094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.302619 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.302623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.304175 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.304187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.305127 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.305131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.305244 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.305283 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 09:43:15.305401 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 09:43:15.305522 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":73,"thread_start_us":81,"threads_started":1}
I20260812 09:43:15.305900 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.299827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.3.147.2:46085)
0812 09:43:15.301334 (+ 1507us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.301354 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.301373 (+ 19us) server_negotiation.cc:407] Connection header received
0812 09:43:15.301473 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.301478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.301666 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.301833 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:15.302061 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.302972 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.304493 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.305304 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.305418 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.305542 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.305648 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.305755 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1396,"thread_start_us":149,"threads_started":1}
I20260812 09:43:15.305950 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.300413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.3.147.3:41343)
0812 09:43:15.300988 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.301002 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.301023 (+ 21us) server_negotiation.cc:407] Connection header received
0812 09:43:15.301103 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.301107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.301299 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.301450 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:15.301723 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.302790 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.304292 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.305099 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.305227 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.305369 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.305729 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.305812 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":149,"threads_started":1}
I20260812 09:43:15.305928 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.299917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:41343 (local address 127.0.0.1:33982)
0812 09:43:15.300257 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.300273 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:15.300375 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:15.301450 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:15.301453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:15.301473 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:15.301587 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.301594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.302919 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.302925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.304120 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.304129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.305439 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.305442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.305532 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.305557 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 09:43:15.305636 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 09:43:15.305787 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":131,"thread_start_us":100,"threads_started":1}
I20260812 09:43:15.306299 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.299638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46085 (local address 127.0.0.1:42670)
0812 09:43:15.299953 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.299968 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:15.300057 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:15.301791 (+ 1734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:15.301794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:15.301826 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:15.301941 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.301949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.303166 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.303170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.304364 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.304373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.305414 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.305418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.305504 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.305527 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:43:15.305620 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 09:43:15.305709 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":7,"thread_start_us":115,"threads_started":1}
I20260812 09:43:15.306718 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.299300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.3.147.1:46269)
0812 09:43:15.300328 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.300352 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.300379 (+ 27us) server_negotiation.cc:407] Connection header received
0812 09:43:15.300466 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.300471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.300711 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.300904 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:15.301266 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.302462 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.304335 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.304980 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.305129 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.305287 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.306468 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.306574 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":81,"threads_started":1}
I20260812 09:43:15.309293 21718 tablet_service.cc:1467] Processing CreateTablet for tablet 420da8486a1149f18e86cfda0b7a21d1 (DEFAULT_TABLE table=test-table [id=4439408315604197a64cb53a54fd1926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:43:15.309545 21564 tablet_service.cc:1467] Processing CreateTablet for tablet 420da8486a1149f18e86cfda0b7a21d1 (DEFAULT_TABLE table=test-table [id=4439408315604197a64cb53a54fd1926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:43:15.309545 21873 tablet_service.cc:1467] Processing CreateTablet for tablet 420da8486a1149f18e86cfda0b7a21d1 (DEFAULT_TABLE table=test-table [id=4439408315604197a64cb53a54fd1926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:43:15.310416 21718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420da8486a1149f18e86cfda0b7a21d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:43:15.310634 21564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420da8486a1149f18e86cfda0b7a21d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:43:15.310639 21873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420da8486a1149f18e86cfda0b7a21d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:43:15.316633 21969 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Bootstrap starting.
I20260812 09:43:15.318670 21971 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Bootstrap starting.
I20260812 09:43:15.319909 21969 tablet_bootstrap.cc:654] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Neither blocks nor log segments found. Creating new log.
I20260812 09:43:15.320494 21970 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Bootstrap starting.
I20260812 09:43:15.320591 21971 tablet_bootstrap.cc:654] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Neither blocks nor log segments found. Creating new log.
I20260812 09:43:15.320785 21969 log.cc:824] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:15.321934 21971 log.cc:824] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:15.322796 21970 tablet_bootstrap.cc:654] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Neither blocks nor log segments found. Creating new log.
I20260812 09:43:15.322902 21969 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: No bootstrap required, opened a new log
I20260812 09:43:15.323148 21969 ts_tablet_manager.cc:1397] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 09:43:15.324209 21971 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: No bootstrap required, opened a new log
I20260812 09:43:15.324178 21970 log.cc:824] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:15.324401 21971 ts_tablet_manager.cc:1397] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:43:15.326232 21970 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: No bootstrap required, opened a new log
I20260812 09:43:15.326431 21970 ts_tablet_manager.cc:1397] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 09:43:15.326400 21969 raft_consensus.cc:348] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:15.326699 21969 raft_consensus.cc:374] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:43:15.326776 21969 raft_consensus.cc:729] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d95759bf4a4641a2c745da8567abe6, State: Initialized, Role: FOLLOWER
I20260812 09:43:15.327241 21969 consensus_queue.cc:260] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:15.327929 21794 heartbeater.cc:503] Master 127.3.147.62:38965 was elected leader, sending a full tablet report...
I20260812 09:43:15.328568 21971 raft_consensus.cc:348] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:15.328840 21971 raft_consensus.cc:374] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:43:15.328943 21971 raft_consensus.cc:729] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7e98c354fe84854a31b3e7a00408844, State: Initialized, Role: FOLLOWER
I20260812 09:43:15.329572 21969 ts_tablet_manager.cc:1428] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:43:15.329555 21971 consensus_queue.cc:260] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:15.330583 21970 raft_consensus.cc:348] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:15.330893 21970 raft_consensus.cc:374] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:43:15.330973 21971 ts_tablet_manager.cc:1428] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:43:15.331015 21970 raft_consensus.cc:729] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c989ce412a497c90399a4da816b383, State: Initialized, Role: FOLLOWER
I20260812 09:43:15.331530 21641 heartbeater.cc:503] Master 127.3.147.62:38965 was elected leader, sending a full tablet report...
I20260812 09:43:15.331441 21970 consensus_queue.cc:260] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:15.332191 21950 heartbeater.cc:503] Master 127.3.147.62:38965 was elected leader, sending a full tablet report...
I20260812 09:43:15.333132 21970 ts_tablet_manager.cc:1428] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260812 09:43:15.508665 21951 tablet.cc:2369] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:43:15.527993 21978 raft_consensus.cc:482] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:15.528281 21978 raft_consensus.cc:504] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:15.529597 21978 leader_election.cc:288] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269), 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085)
I20260812 09:43:15.534226 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.530179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:51881 (local address 127.3.147.1:46269)
0812 09:43:15.530323 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.530329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.530345 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:43:15.530401 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.530404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.530466 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.530561 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:15.530919 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.531716 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.533062 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.533632 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.533767 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.533838 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.534032 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.534121 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 09:43:15.534226 21952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.529822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:46269 (local address 127.3.147.3:51881)
0812 09:43:15.529990 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.530008 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:15.530120 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:15.530579 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:15.530581 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:15.530600 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:15.530776 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.530785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.531850 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.531856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.532909 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.532920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.533770 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.533774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.533888 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.533915 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 09:43:15.534013 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 09:43:15.534090 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":1}
I20260812 09:43:15.534708 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.529828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46085 (local address 127.3.147.3:34685)
0812 09:43:15.530494 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.530507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:15.530663 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:15.531015 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:15.531018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:15.531034 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:15.531164 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.531169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.531931 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.531934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.533227 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:15.533238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.534311 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.534315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:15.534411 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.534434 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 09:43:15.534496 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 09:43:15.534564 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":361,"thread_start_us":115,"threads_started":1}
I20260812 09:43:15.534780 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.529973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:34685 (local address 127.3.147.2:46085)
0812 09:43:15.530118 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.530123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.530584 (+ 461us) server_negotiation.cc:407] Connection header received
0812 09:43:15.530823 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.530829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.530905 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.531004 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:15.531290 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.531812 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.533378 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.534058 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.534174 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.534235 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.534554 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.534652 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 09:43:15.534955 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420da8486a1149f18e86cfda0b7a21d1" candidate_uuid: "09c989ce412a497c90399a4da816b383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7e98c354fe84854a31b3e7a00408844" is_pre_election: true
I20260812 09:43:15.535223 21586 raft_consensus.cc:2454] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09c989ce412a497c90399a4da816b383 in term 0.
I20260812 09:43:15.535710 21821 leader_election.cc:302] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c989ce412a497c90399a4da816b383, e7e98c354fe84854a31b3e7a00408844; no voters:
I20260812 09:43:15.535993 21978 raft_consensus.cc:2781] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:43:15.536098 21978 raft_consensus.cc:482] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:43:15.536157 21978 raft_consensus.cc:3037] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:43:15.536263 21742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420da8486a1149f18e86cfda0b7a21d1" candidate_uuid: "09c989ce412a497c90399a4da816b383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d95759bf4a4641a2c745da8567abe6" is_pre_election: true
I20260812 09:43:15.536556 21742 raft_consensus.cc:2454] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09c989ce412a497c90399a4da816b383 in term 0.
I20260812 09:43:15.537528 21978 raft_consensus.cc:504] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:15.537884 21978 leader_election.cc:288] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [CANDIDATE]: Term 1 election: Requested vote from peers e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269), 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085)
I20260812 09:43:15.538261 21742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420da8486a1149f18e86cfda0b7a21d1" candidate_uuid: "09c989ce412a497c90399a4da816b383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d95759bf4a4641a2c745da8567abe6"
I20260812 09:43:15.538271 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420da8486a1149f18e86cfda0b7a21d1" candidate_uuid: "09c989ce412a497c90399a4da816b383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7e98c354fe84854a31b3e7a00408844"
I20260812 09:43:15.538410 21742 raft_consensus.cc:3037] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:43:15.538410 21586 raft_consensus.cc:3037] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:43:15.539520 21742 raft_consensus.cc:2454] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09c989ce412a497c90399a4da816b383 in term 1.
I20260812 09:43:15.539642 21586 raft_consensus.cc:2454] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09c989ce412a497c90399a4da816b383 in term 1.
I20260812 09:43:15.539826 21823 leader_election.cc:302] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c989ce412a497c90399a4da816b383, 36d95759bf4a4641a2c745da8567abe6; no voters:
I20260812 09:43:15.540009 21978 raft_consensus.cc:2781] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:43:15.540268 21978 raft_consensus.cc:686] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 1 LEADER]: Becoming Leader. State: Replica: 09c989ce412a497c90399a4da816b383, State: Running, Role: LEADER
I20260812 09:43:15.540582 21978 consensus_queue.cc:237] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
W20260812 09:43:15.541844 21796 tablet.cc:2369] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:43:15.544011 21420 catalog_manager.cc:5495] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09c989ce412a497c90399a4da816b383 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "09c989ce412a497c90399a4da816b383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } health_report { overall_health: UNKNOWN } } }
W20260812 09:43:15.567008 21642 tablet.cc:2369] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:43:15.599363 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.595436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.3.147.3:41343)
0812 09:43:15.595606 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:15.595611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:15.595626 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:43:15.595790 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:15.595793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:15.595855 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:15.595976 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:43:15.596293 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.596907 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.597812 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:15.598038 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:15.598154 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:15.598491 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:43:15.598927 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:43:15.599059 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:15.599125 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 09:43:15.599194 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 09:43:15.605831 21742 raft_consensus.cc:1261] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 1 FOLLOWER]: Refusing update from remote peer 09c989ce412a497c90399a4da816b383: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:43:15.605859 21586 raft_consensus.cc:1261] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Refusing update from remote peer 09c989ce412a497c90399a4da816b383: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:43:15.606608 21980 consensus_queue.cc:1035] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:43:15.606834 21978 consensus_queue.cc:1035] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:43:15.613149 21984 mvcc.cc:205] Tried to move back new op lower bound from 7317617850793680896 to 7317617850537136128. Current Snapshot: MvccSnapshot[applied={T|T < 7317617850793680896}]
I20260812 09:43:15.617215 21987 mvcc.cc:205] Tried to move back new op lower bound from 7317617850793680896 to 7317617850537136128. Current Snapshot: MvccSnapshot[applied={T|T < 7317617850793680896}]
I20260812 09:43:15.624694 21421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:43:15.624848 21421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:43:15.627197 21421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Sending DeleteTablet for 3 replicas of tablet 420da8486a1149f18e86cfda0b7a21d1
I20260812 09:43:15.628041 21718 tablet_service.cc:1514] Processing DeleteTablet for tablet 420da8486a1149f18e86cfda0b7a21d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:43:15 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:42670
I20260812 09:43:15.628091 21564 tablet_service.cc:1514] Processing DeleteTablet for tablet 420da8486a1149f18e86cfda0b7a21d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:43:15 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:46358
I20260812 09:43:15.628229 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 21495
I20260812 09:43:15.628203 21873 tablet_service.cc:1514] Processing DeleteTablet for tablet 420da8486a1149f18e86cfda0b7a21d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:43:15 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:33982
I20260812 09:43:15.633105 21997 tablet_replica.cc:330] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: stopping tablet replica
I20260812 09:43:15.633458 21997 raft_consensus.cc:2229] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:43:15.633793 21997 raft_consensus.cc:2258] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:43:15.634711 21998 tablet_replica.cc:330] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: stopping tablet replica
I20260812 09:43:15.635048 21998 raft_consensus.cc:2229] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:43:15.635424 21997 ts_tablet_manager.cc:1905] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:43:15.635511 21998 raft_consensus.cc:2258] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:43:15.636924 21998 ts_tablet_manager.cc:1905] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:43:15.639038 21997 ts_tablet_manager.cc:1918] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:43:15.639231 21997 log.cc:1196] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/420da8486a1149f18e86cfda0b7a21d1
I20260812 09:43:15.639654 21997 ts_tablet_manager.cc:1939] T 420da8486a1149f18e86cfda0b7a21d1 P 36d95759bf4a4641a2c745da8567abe6: Deleting consensus metadata
W20260812 09:43:15.640374 21403 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:46269 (error 108)
I20260812 09:43:15.640642 21998 ts_tablet_manager.cc:1918] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:43:15.640841 21998 log.cc:1196] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/420da8486a1149f18e86cfda0b7a21d1
I20260812 09:43:15.641261 21998 ts_tablet_manager.cc:1939] T 420da8486a1149f18e86cfda0b7a21d1 P 09c989ce412a497c90399a4da816b383: Deleting consensus metadata
I20260812 09:43:15.641305 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 21646
I20260812 09:43:15.643049 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.641590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:46269 (local address 127.0.0.1:46364)
0812 09:43:15.642896 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.642965 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:46269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1222}
W20260812 09:43:15.644665 21403 catalog_manager.cc:4584] TS e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 420da8486a1149f18e86cfda0b7a21d1: Network error: Client connection negotiation failed: client connection to 127.3.147.1:46269: connect: Connection refused (error 111)
I20260812 09:43:15.652881 21403 catalog_manager.cc:4841] TS 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343): tablet 420da8486a1149f18e86cfda0b7a21d1 (table test-table [id=4439408315604197a64cb53a54fd1926]) successfully deleted
I20260812 09:43:15.658385 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 21802
I20260812 09:43:15.658421 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:15.658122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46085 (local address 127.0.0.1:42684)
0812 09:43:15.658266 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:15.658329 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.2:46085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 09:43:15.665864 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 21386
I20260812 09:43:35.675887 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38965
--webserver_interface=127.3.147.62
--webserver_port=37671
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:38965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:43:35.793606 22000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:35.793882 22000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:35.793952 22000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:35.797974 22000 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 09:43:35.798034 22000 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 09:43:35.798072 22000 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 09:43:35.798090 22000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:43:35.798171 22000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:35.798209 22000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:35.798254 22000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:43:35.798308 22000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:43:35.803654 22000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:38965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:35.804740 22000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:35.805811 22000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:35.820127 22006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:35.820259 22000 server_base.cc:1034] running on GCE node
W20260812 09:43:35.820176 22009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:35.820199 22007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:35.820997 22000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:35.822160 22000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:35.823343 22000 hybrid_clock.cc:648] HybridClock initialized: now 1786527815823313 us; error 41 us; skew 500 ppm
Aug 12 09:43:35 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527815, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:35.831411 22000 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:43:35.832436 22000 webserver.cc:460] Webserver started at http://127.3.147.62:37671/ using document root <none> and password file <none>
I20260812 09:43:35.833083 22000 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:35.833200 22000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:35.837280 22000 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:43:35.839480 22018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:35.840703 22000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:43:35.840845 22000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "e35011e296454af99c25eba73ae2103b"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "6508ba3ecf5162d3f591099791cd9887"
server_key_iv: "b440fea6347ec89523eb3a37a9438881"
server_key_version: "encryptionkey@0"
I20260812 09:43:35.841449 22000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:35.871821 22000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:35.875031 22000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:35.875249 22000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:35.883006 22000 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:38965
I20260812 09:43:35.883328 22077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:38965 every 8 connection(s)
I20260812 09:43:35.884254 22000 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:43:35.888821 22078 sys_catalog.cc:263] Verifying existing consensus state
I20260812 09:43:35.890547 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Bootstrap starting.
I20260812 09:43:35.894625 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 22000
I20260812 09:43:35.895139 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:46269
--local_ip_for_outbound_sockets=127.3.147.1
--tserver_master_addrs=127.3.147.62:38965
--webserver_port=33233
--webserver_interface=127.3.147.1
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:43:35.909124 22078 log.cc:824] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:35.917878 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:43:35.918252 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Bootstrap complete.
I20260812 09:43:35.922369 22078 raft_consensus.cc:348] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:35.922765 22078 raft_consensus.cc:729] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35011e296454af99c25eba73ae2103b, State: Initialized, Role: FOLLOWER
I20260812 09:43:35.923235 22078 consensus_queue.cc:260] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:35.923380 22078 raft_consensus.cc:388] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:43:35.923466 22078 raft_consensus.cc:482] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:43:35.923583 22078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 1 FOLLOWER]: Advancing to term 2
I20260812 09:43:35.925557 22078 raft_consensus.cc:504] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:35.925899 22078 leader_election.cc:302] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e35011e296454af99c25eba73ae2103b; no voters:
I20260812 09:43:35.926254 22078 leader_election.cc:288] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 09:43:35.926339 22082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 2 FOLLOWER]: Leader election won for term 2
I20260812 09:43:35.926613 22082 raft_consensus.cc:686] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 2 LEADER]: Becoming Leader. State: Replica: e35011e296454af99c25eba73ae2103b, State: Running, Role: LEADER
I20260812 09:43:35.926966 22082 consensus_queue.cc:237] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:35.927521 22078 sys_catalog.cc:564] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:43:35.927980 22082 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35011e296454af99c25eba73ae2103b. Latest consensus state: current_term: 2 leader_uuid: "e35011e296454af99c25eba73ae2103b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } } }
I20260812 09:43:35.928092 22082 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: This master's current role is: LEADER
I20260812 09:43:35.928573 22083 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e35011e296454af99c25eba73ae2103b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } } }
I20260812 09:43:35.928763 22083 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: This master's current role is: LEADER
I20260812 09:43:35.933187 22102 catalog_manager.cc:1259] Loaded cluster ID: 660e6563a012411eb50796cc9b8ec33a
I20260812 09:43:35.933250 22102 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: loading cluster ID for follower catalog manager: success
I20260812 09:43:35.935724 22102 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: acquiring CA information for follower catalog manager: success
I20260812 09:43:35.937318 22102 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 09:43:35.937461 22088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:43:35.939419 22088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:43:35.939936 22088 catalog_manager.cc:1259] Loaded cluster ID: 660e6563a012411eb50796cc9b8ec33a
I20260812 09:43:35.939996 22088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:43:35.940708 22088 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:43:35.941207 22088 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Loaded TSK: 0
I20260812 09:43:35.955580 22088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Generated new TSK 1
I20260812 09:43:35.955866 22088 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 09:43:36.015789 22080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:36.016089 22080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:36.016213 22080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:36.020797 22080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:36.020846 22080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:36.020931 22080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:43:36.026079 22080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:46269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=33233
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:36.027022 22080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:36.028088 22080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:36.039047 22113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:36.039182 22115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:36.039041 22112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:36.039494 22080 server_base.cc:1034] running on GCE node
I20260812 09:43:36.039937 22080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:36.040756 22080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:36.041968 22080 hybrid_clock.cc:648] HybridClock initialized: now 1786527816041917 us; error 62 us; skew 500 ppm
Aug 12 09:43:36 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527816, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:36.045588 22080 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:43:36.046607 22080 webserver.cc:460] Webserver started at http://127.3.147.1:33233/ using document root <none> and password file <none>
I20260812 09:43:36.047200 22080 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:36.047323 22080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:36.051561 22080 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:43:36.054121 22122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:36.055567 22080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:43:36.055714 22080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "e7e98c354fe84854a31b3e7a00408844"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "639c9edb9e48c4b4c8da861153ffd740"
server_key_iv: "295ed69c2bb43cc2b5706102438e41c9"
server_key_version: "encryptionkey@0"
I20260812 09:43:36.056361 22080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:36.068667 22080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:36.071883 22080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:36.072136 22080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:36.072804 22080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:36.073993 22131 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 09:43:36.078295 22080 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 09:43:36.078357 22080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260812 09:43:36.078426 22080 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 09:43:36.080137 22080 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 09:43:36.080222 22080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260812 09:43:36.080474 22131 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Bootstrap starting.
I20260812 09:43:36.091655 22080 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:46269
I20260812 09:43:36.091876 22251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:46269 every 8 connection(s)
I20260812 09:43:36.093080 22080 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:43:36.102830 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 22080
I20260812 09:43:36.103394 22131 log.cc:824] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:36.103404 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:46085
--local_ip_for_outbound_sockets=127.3.147.2
--tserver_master_addrs=127.3.147.62:38965
--webserver_port=46141
--webserver_interface=127.3.147.2
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:43:36 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527816, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:43:36.111339 22131 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:43:36.111796 22131 tablet_bootstrap.cc:492] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Bootstrap complete.
I20260812 09:43:36.113092 22131 ts_tablet_manager.cc:1397] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Time spent bootstrapping tablet: real 0.033s user 0.015s sys 0.016s
I20260812 09:43:36.114938 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.097244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:52633 (local address 127.3.147.62:38965)
0812 09:43:36.097674 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.097694 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.098141 (+ 447us) server_negotiation.cc:407] Connection header received
0812 09:43:36.099327 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.099338 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.099636 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.099983 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:36.101241 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.102245 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.104056 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.104704 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.104895 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.108112 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:36.108121 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:36.108179 (+ 58us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:36.108218 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:36.110595 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.111332 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.111341 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.111349 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.111526 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.112022 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.112028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.112032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.112572 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:36.112691 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.113548 (+ 857us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.113895 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":112,"threads_started":1}
I20260812 09:43:36.115427 22131 raft_consensus.cc:348] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:36.115899 22131 raft_consensus.cc:729] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7e98c354fe84854a31b3e7a00408844, State: Initialized, Role: FOLLOWER
I20260812 09:43:36.116469 22131 consensus_queue.cc:260] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } }
I20260812 09:43:36.116681 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.097303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:52633)
0812 09:43:36.097941 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.097998 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.098888 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.100044 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.100053 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.100538 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:36.101019 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.101034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.102415 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.102419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.103882 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.103898 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.104864 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.104869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.105079 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.105854 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:36.105881 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:36.107887 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:36.110758 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:36.110764 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:36.110779 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:36.111176 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:36.111692 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:36.111698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:36.111701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:36.111894 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:36.112797 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:36.112805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:36.113137 (+ 332us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.115269 (+ 2132us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.115631 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":157,"threads_started":1}
I20260812 09:43:36.117674 22131 ts_tablet_manager.cc:1428] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 09:43:36.119428 22252 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38965
I20260812 09:43:36.119673 22252 heartbeater.cc:464] Registering TS with master...
I20260812 09:43:36.120208 22252 heartbeater.cc:511] Master 127.3.147.62:38965 requested a full tablet report, sending...
I20260812 09:43:36.122718 22036 ts_manager.cc:194] Registered new tserver with Master: e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269)
I20260812 09:43:36.126724 22036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:52633
I20260812 09:43:36.129619 22252 heartbeater.cc:503] Master 127.3.147.62:38965 was elected leader, sending a full tablet report...
Aug 12 09:43:36 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527815, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.1@KRBTEST.COM
I20260812 09:43:36.137655 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.126024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:46269 (local address 127.0.0.1:35894)
0812 09:43:36.126609 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.126657 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.126788 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.128217 (+ 1429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.128224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.128675 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:36.128815 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.128823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.130166 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.130169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.131184 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.131196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.131745 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.131749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.131852 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.132439 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:36.132454 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:36.133963 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:36.135973 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:36.135982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:36.135986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:36.136299 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:36.136609 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:36.136611 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:36.136613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:36.136691 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:36.137191 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:36.137195 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:36.137289 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.137388 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.137519 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":127,"threads_started":1}
I20260812 09:43:36.137843 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.126090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.3.147.1:46269)
0812 09:43:36.127721 (+ 1631us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.127732 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.127752 (+ 20us) server_negotiation.cc:407] Connection header received
0812 09:43:36.127809 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.127835 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.128023 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.128187 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:36.128950 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.130027 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.131303 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.131628 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.131713 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.134101 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:36.134112 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:36.134124 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:36.134170 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:36.135859 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.136409 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.136415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.136417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.136511 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.136786 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.136789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.136791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.137075 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:36.137210 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.137508 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.137672 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1363,"thread_start_us":113,"threads_started":1}
I20260812 09:43:36.138868 22177 tablet_service.cc:1514] Processing DeleteTablet for tablet 420da8486a1149f18e86cfda0b7a21d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:43:15 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35894
I20260812 09:43:36.139822 22261 tablet_replica.cc:330] stopping tablet replica
I20260812 09:43:36.140102 22261 raft_consensus.cc:2229] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:43:36.140369 22261 raft_consensus.cc:2258] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:43:36.141342 22261 ts_tablet_manager.cc:1905] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:43:36.145009 22261 ts_tablet_manager.cc:1918] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:43:36.145128 22261 log.cc:1196] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal/wals/420da8486a1149f18e86cfda0b7a21d1
I20260812 09:43:36.145428 22261 ts_tablet_manager.cc:1939] T 420da8486a1149f18e86cfda0b7a21d1 P e7e98c354fe84854a31b3e7a00408844: Deleting consensus metadata
I20260812 09:43:36.147305 22019 catalog_manager.cc:4841] TS e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269): tablet 420da8486a1149f18e86cfda0b7a21d1 (table test-table [id=4439408315604197a64cb53a54fd1926]) successfully deleted
W20260812 09:43:36.227187 22257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:36.227473 22257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:36.227553 22257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:36.231947 22257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:36.232026 22257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:36.232112 22257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:43:36.237962 22257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:46085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=46141
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:36.239156 22257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:36.240309 22257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:36.250850 22268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:36.250991 22267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:36.251143 22257 server_base.cc:1034] running on GCE node
W20260812 09:43:36.251188 22270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:36.251693 22257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:36.252440 22257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:36.253659 22257 hybrid_clock.cc:648] HybridClock initialized: now 1786527816253639 us; error 36 us; skew 500 ppm
Aug 12 09:43:36 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527816, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:36.256784 22257 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:43:36.257771 22257 webserver.cc:460] Webserver started at http://127.3.147.2:46141/ using document root <none> and password file <none>
I20260812 09:43:36.258435 22257 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:36.258590 22257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:36.263105 22257 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:43:36.265529 22277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:36.266809 22257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:43:36.266970 22257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "36d95759bf4a4641a2c745da8567abe6"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "8995e397b4b9773811ed5d7b1916f86a"
server_key_iv: "9a4f20be0dbb19ae0126b447ea2913fa"
server_key_version: "encryptionkey@0"
I20260812 09:43:36.267637 22257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:36.294319 22257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:36.298296 22257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:36.298543 22257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:36.299257 22257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:36.300326 22257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:43:36.300408 22257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:36.300493 22257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:43:36.300565 22257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:36.311620 22257 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46085
I20260812 09:43:36.311645 22405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46085 every 8 connection(s)
I20260812 09:43:36.312832 22257 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:43:36.322132 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 22257
I20260812 09:43:36.322567 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:41343
--local_ip_for_outbound_sockets=127.3.147.3
--tserver_master_addrs=127.3.147.62:38965
--webserver_port=45151
--webserver_interface=127.3.147.3
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 09:43:36 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527816, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:43:36.328356 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.314777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:45473 (local address 127.3.147.62:38965)
0812 09:43:36.314934 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.314938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.315790 (+ 852us) server_negotiation.cc:407] Connection header received
0812 09:43:36.317153 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.317157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.317214 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.317330 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:36.318507 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.319052 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.320212 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.320482 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.320598 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.323480 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:36.323487 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:36.323493 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:36.323529 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:36.325690 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.326301 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.326306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.326308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.326428 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.326912 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.326915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.326916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.327190 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:36.327296 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.327928 (+ 632us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.328098 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 09:43:36.329433 22408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.315174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:45473)
0812 09:43:36.315635 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.315680 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.316937 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.317503 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.317509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.317867 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:36.318357 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.318371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.319198 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.319202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.320052 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.320061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.320637 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.320642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.320831 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.321402 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:36.321417 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:36.323296 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:36.325876 (+ 2580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:36.325883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:36.325915 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:36.326163 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:36.326551 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:36.326558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:36.326563 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:36.326775 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:36.327318 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:36.327327 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:36.327646 (+ 319us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.327875 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.328297 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":133,"threads_started":1}
I20260812 09:43:36.330797 22406 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38965
I20260812 09:43:36.331074 22406 heartbeater.cc:464] Registering TS with master...
I20260812 09:43:36.331756 22406 heartbeater.cc:511] Master 127.3.147.62:38965 requested a full tablet report, sending...
I20260812 09:43:36.333037 22036 ts_manager.cc:194] Registered new tserver with Master: 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085)
I20260812 09:43:36.333640 22036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:45473
W20260812 09:43:36.458137 22409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:36.458398 22409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:36.458472 22409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:36.462568 22409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:36.462625 22409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:36.462721 22409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:43:36.467975 22409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:41343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=45151
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:36.469053 22409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:36.470106 22409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:36.480192 22417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:36.480209 22416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:36.480391 22419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:36.480517 22409 server_base.cc:1034] running on GCE node
I20260812 09:43:36.480960 22409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:36.481576 22409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:36.482738 22409 hybrid_clock.cc:648] HybridClock initialized: now 1786527816482722 us; error 28 us; skew 500 ppm
Aug 12 09:43:36 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527816, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:36.485500 22409 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:43:36.486418 22409 webserver.cc:460] Webserver started at http://127.3.147.3:45151/ using document root <none> and password file <none>
I20260812 09:43:36.486979 22409 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:36.487076 22409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:36.490926 22409 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:43:36.492959 22428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:36.494158 22409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 09:43:36.494290 22409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "09c989ce412a497c90399a4da816b383"
format_stamp: "Formatted at 2026-08-12 09:43:15 on dist-test-slave-cbbz"
server_key: "b3fa0d649b5ccf90746f787afeda80a4"
server_key_iv: "852079846513fb407a96bbbcec1377c6"
server_key_version: "encryptionkey@0"
I20260812 09:43:36.494849 22409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:36.520048 22409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:36.523051 22409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:36.523259 22409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:36.523886 22409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:36.524786 22409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:43:36.524864 22409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:36.524940 22409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:43:36.524983 22409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:36.534859 22409 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:41343
I20260812 09:43:36.534896 22555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:41343 every 8 connection(s)
I20260812 09:43:36.535853 22409 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:43:36.541848 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 22409
Aug 12 09:43:36 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527816, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:43:36.550985 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.537744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:42437 (local address 127.3.147.62:38965)
0812 09:43:36.537883 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.537887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.538744 (+ 857us) server_negotiation.cc:407] Connection header received
0812 09:43:36.540144 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.540177 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.540268 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.540364 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:36.541566 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.542167 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.543850 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.544171 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.544273 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.546980 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:36.546986 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:36.546990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:36.547022 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:36.548769 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.549266 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.549272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.549274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.549361 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.549768 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.549771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.549772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.549958 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:36.550023 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.550653 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.550807 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:43:36.550987 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.542549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45646 (local address 127.3.147.62:38965)
0812 09:43:36.542815 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.542818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.542961 (+ 143us) server_negotiation.cc:407] Connection header received
0812 09:43:36.543089 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.543092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.543128 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.543204 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:36.544086 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.544730 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.545598 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.545925 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.546026 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.547585 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:36.547601 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:36.547605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:36.547647 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:36.549182 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.549672 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.549674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.549675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.549763 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.550155 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.550157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.550159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.550315 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:36.550455 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.550673 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.550869 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260812 09:43:36.552019 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.538106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:42437)
0812 09:43:36.538577 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.538620 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.539890 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.540561 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.540568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.540948 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:36.541418 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.541431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.542289 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.542292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.543712 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.543725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.544322 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.544325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.544460 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.545126 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:36.545142 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:36.546808 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:36.548891 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:36.548896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:36.548907 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:36.549169 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:36.549463 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:36.549467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:36.549472 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:36.549672 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:36.550076 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:36.550083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:36.550352 (+ 269us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.550588 (+ 236us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.550927 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":131,"threads_started":1}
I20260812 09:43:36.553236 22556 heartbeater.cc:347] Connected to a master server at 127.3.147.62:38965
I20260812 09:43:36.553496 22556 heartbeater.cc:464] Registering TS with master...
I20260812 09:43:36.553982 22556 heartbeater.cc:511] Master 127.3.147.62:38965 requested a full tablet report, sending...
I20260812 09:43:36.554956 22036 ts_manager.cc:194] Registered new tserver with Master: 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343)
I20260812 09:43:36.555508 22036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:42437
I20260812 09:43:36.564992 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:43:36.569013 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.565678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45656 (local address 127.3.147.62:38965)
0812 09:43:36.565815 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.565824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.566063 (+ 239us) server_negotiation.cc:407] Connection header received
0812 09:43:36.566162 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.566164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.566266 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.566359 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:43:36.566573 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.567079 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.567915 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.568205 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.568305 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.568570 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:43:36.568704 (+ 134us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 09:43:36.568903 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45656: token expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 09:43:36.569103 22254 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45656: token expired
W20260812 09:43:36.569523 21953 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.147.62:38965)
I20260812 09:43:36.576764 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.569882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.3.147.62:38965)
0812 09:43:36.569994 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.569997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.570051 (+ 54us) server_negotiation.cc:407] Connection header received
0812 09:43:36.570134 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.570136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.570171 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.570235 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:36.570837 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.571331 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.572114 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.572396 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.572478 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.573749 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:36.573752 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:36.573754 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:36.573776 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:36.575067 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.575614 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.575617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.575618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.575696 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:36.575994 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:36.575996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:36.575997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:36.576227 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:36.576333 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.576550 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.576651 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 09:43:36.579620 22036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:43:36.582963 22036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:43:36.590972 22177 tablet_service.cc:1467] Processing CreateTablet for tablet 4df0bdc9ee564669bc1a182585cfadf0 (DEFAULT_TABLE table=test-table [id=8bd52df96ad64e27a4ce77fdb457e044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:43:36.591729 22177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4df0bdc9ee564669bc1a182585cfadf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:43:36.594566 22131 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Bootstrap starting.
I20260812 09:43:36.595485 22131 tablet_bootstrap.cc:654] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Neither blocks nor log segments found. Creating new log.
I20260812 09:43:36.595645 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.590076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:41343 (local address 127.0.0.1:44122)
0812 09:43:36.590293 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.590307 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.590397 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.591431 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.591434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.591451 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:36.591548 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.591554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.593048 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.593051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.593872 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.593881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.595137 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.595142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.595248 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.595310 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.595388 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.595466 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":23,"thread_start_us":99,"threads_started":1}
I20260812 09:43:36.595834 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.589646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46085 (local address 127.0.0.1:36044)
0812 09:43:36.589823 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.589838 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.589915 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.591191 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.591194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.591213 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:36.591326 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.591332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.593012 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.593016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.594226 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.594239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.595347 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.595350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.595481 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.595510 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.595585 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.595692 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":21}
I20260812 09:43:36.596187 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.590274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44122 (local address 127.3.147.3:41343)
0812 09:43:36.590637 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.590651 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.590678 (+ 27us) server_negotiation.cc:407] Connection header received
0812 09:43:36.590765 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.590772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.590991 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.591175 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:36.591700 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.592879 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.594008 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.594796 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.594902 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.595069 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.595881 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.595997 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":141,"threads_started":1}
I20260812 09:43:36.596541 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.590109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.3.147.2:46085)
0812 09:43:36.590727 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.590738 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.590757 (+ 19us) server_negotiation.cc:407] Connection header received
0812 09:43:36.590822 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.590827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.590990 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.591289 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:36.591499 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.592877 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.594403 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.595191 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.596059 (+ 868us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.596250 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.596308 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.596400 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":142,"threads_started":1}
I20260812 09:43:36.597175 22131 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: No bootstrap required, opened a new log
I20260812 09:43:36.597277 22131 ts_tablet_manager.cc:1397] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 09:43:36.597746 22131 raft_consensus.cc:348] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.597867 22131 raft_consensus.cc:374] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:43:36.597905 22131 raft_consensus.cc:729] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7e98c354fe84854a31b3e7a00408844, State: Initialized, Role: FOLLOWER
I20260812 09:43:36.598060 22131 consensus_queue.cc:260] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.598800 22131 ts_tablet_manager.cc:1428] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 09:43:36.599378 22330 tablet_service.cc:1467] Processing CreateTablet for tablet 4df0bdc9ee564669bc1a182585cfadf0 (DEFAULT_TABLE table=test-table [id=8bd52df96ad64e27a4ce77fdb457e044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:43:36.599399 22476 tablet_service.cc:1467] Processing CreateTablet for tablet 4df0bdc9ee564669bc1a182585cfadf0 (DEFAULT_TABLE table=test-table [id=8bd52df96ad64e27a4ce77fdb457e044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:43:36.600528 22330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4df0bdc9ee564669bc1a182585cfadf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:43:36.600545 22476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4df0bdc9ee564669bc1a182585cfadf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:43:36.607369 22572 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: Bootstrap starting.
I20260812 09:43:36.607437 22571 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: Bootstrap starting.
I20260812 09:43:36.609493 22571 tablet_bootstrap.cc:654] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: Neither blocks nor log segments found. Creating new log.
I20260812 09:43:36.609880 22572 tablet_bootstrap.cc:654] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: Neither blocks nor log segments found. Creating new log.
I20260812 09:43:36.610646 22572 log.cc:824] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:36.610683 22571 log.cc:824] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:36.612710 22572 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: No bootstrap required, opened a new log
I20260812 09:43:36.612746 22571 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: No bootstrap required, opened a new log
I20260812 09:43:36.612928 22572 ts_tablet_manager.cc:1397] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:43:36.612936 22571 ts_tablet_manager.cc:1397] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 09:43:36.615733 22571 raft_consensus.cc:348] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.615734 22572 raft_consensus.cc:348] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.615978 22571 raft_consensus.cc:374] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:43:36.615972 22572 raft_consensus.cc:374] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:43:36.616041 22571 raft_consensus.cc:729] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c989ce412a497c90399a4da816b383, State: Initialized, Role: FOLLOWER
I20260812 09:43:36.616050 22572 raft_consensus.cc:729] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d95759bf4a4641a2c745da8567abe6, State: Initialized, Role: FOLLOWER
I20260812 09:43:36.616493 22571 consensus_queue.cc:260] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.616493 22572 consensus_queue.cc:260] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.617188 22406 heartbeater.cc:503] Master 127.3.147.62:38965 was elected leader, sending a full tablet report...
I20260812 09:43:36.617420 22556 heartbeater.cc:503] Master 127.3.147.62:38965 was elected leader, sending a full tablet report...
I20260812 09:43:36.617551 22572 ts_tablet_manager.cc:1428] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 09:43:36.617749 22571 ts_tablet_manager.cc:1428] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 09:43:36.688112 22575 raft_consensus.cc:482] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:36.688457 22575 raft_consensus.cc:504] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.689715 22575 leader_election.cc:288] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343), e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269)
I20260812 09:43:36.691875 22576 raft_consensus.cc:482] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:36.692054 22576 raft_consensus.cc:504] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.694167 22576 leader_election.cc:288] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085), e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269)
I20260812 09:43:36.694559 22408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.689957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:41343 (local address 127.3.147.2:44679)
0812 09:43:36.690087 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.690105 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.690211 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.690610 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.690613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.690632 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:36.690771 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.690780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.691869 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.691872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.693062 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.693074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.694065 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.694072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.694212 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.694237 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.694318 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.694404 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":4}
I20260812 09:43:36.694852 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.690410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:45463 (local address 127.3.147.1:46269)
0812 09:43:36.690686 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.690689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.690703 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:43:36.690849 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.690853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.690922 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.691037 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:36.691293 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.692018 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.693168 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.693895 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.694003 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.694178 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.694628 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.694722 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":83,"threads_started":1}
I20260812 09:43:36.694931 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.690262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:46269 (local address 127.3.147.2:45463)
0812 09:43:36.690618 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.690630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.690721 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.691072 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.691074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.691090 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:36.691183 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.691187 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.692165 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.692168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.693024 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.693034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.694240 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.694243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.694351 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.694378 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.694448 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.694519 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260812 09:43:36.694947 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.690069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:44679 (local address 127.3.147.3:41343)
0812 09:43:36.690237 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.690243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.690262 (+ 19us) server_negotiation.cc:407] Connection header received
0812 09:43:36.690324 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.690330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.690385 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.690485 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:36.690899 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.691721 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.693220 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.693831 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.693935 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.693999 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.694759 (+ 760us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.694845 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:43:36.695520 22504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df0bdc9ee564669bc1a182585cfadf0" candidate_uuid: "36d95759bf4a4641a2c745da8567abe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09c989ce412a497c90399a4da816b383" is_pre_election: true
I20260812 09:43:36.695677 22201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df0bdc9ee564669bc1a182585cfadf0" candidate_uuid: "36d95759bf4a4641a2c745da8567abe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7e98c354fe84854a31b3e7a00408844" is_pre_election: true
I20260812 09:43:36.695755 22504 raft_consensus.cc:2454] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d95759bf4a4641a2c745da8567abe6 in term 0.
I20260812 09:43:36.695957 22201 raft_consensus.cc:2454] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d95759bf4a4641a2c745da8567abe6 in term 0.
I20260812 09:43:36.696341 22278 leader_election.cc:302] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c989ce412a497c90399a4da816b383, 36d95759bf4a4641a2c745da8567abe6; no voters:
I20260812 09:43:36.696607 22575 raft_consensus.cc:2781] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:43:36.696705 22575 raft_consensus.cc:482] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:43:36.696746 22575 raft_consensus.cc:3037] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:43:36.698105 22575 raft_consensus.cc:504] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.698266 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.693840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:56303 (local address 127.3.147.2:46085)
0812 09:43:36.693986 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.693991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.694003 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:43:36.694041 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.694043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.694082 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.694162 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:36.694523 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.695470 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.697056 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.697653 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.697760 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.697811 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.698095 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.698162 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":262,"server-negotiator.queue_time_us":58}
I20260812 09:43:36.698275 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.693410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46085 (local address 127.3.147.3:56303)
0812 09:43:36.693640 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.693654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.693730 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.694227 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.694230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.694248 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:36.694351 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.694356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.695597 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.695600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.696924 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.696938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.697810 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.697814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.697940 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.697980 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.698065 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.698162 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260812 09:43:36.698475 22575 leader_election.cc:288] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [CANDIDATE]: Term 1 election: Requested vote from peers 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343), e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269)
I20260812 09:43:36.698829 22201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df0bdc9ee564669bc1a182585cfadf0" candidate_uuid: "36d95759bf4a4641a2c745da8567abe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7e98c354fe84854a31b3e7a00408844"
I20260812 09:43:36.698968 22201 raft_consensus.cc:3037] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:43:36.698932 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df0bdc9ee564669bc1a182585cfadf0" candidate_uuid: "09c989ce412a497c90399a4da816b383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d95759bf4a4641a2c745da8567abe6" is_pre_election: true
I20260812 09:43:36.698993 22504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df0bdc9ee564669bc1a182585cfadf0" candidate_uuid: "36d95759bf4a4641a2c745da8567abe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09c989ce412a497c90399a4da816b383"
I20260812 09:43:36.699100 22504 raft_consensus.cc:3037] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:43:36.699172 22355 raft_consensus.cc:2379] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09c989ce412a497c90399a4da816b383 in current term 1: Already voted for candidate 36d95759bf4a4641a2c745da8567abe6 in this term.
I20260812 09:43:36.699841 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.693474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:46269 (local address 127.3.147.3:45957)
0812 09:43:36.694487 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.694500 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:36.694589 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:36.696022 (+ 1433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:36.696025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:36.696045 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:43:36.696185 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.696195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.697140 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.697143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.698518 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:36.698530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.699460 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.699464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:36.699556 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.699582 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 09:43:36.699630 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 09:43:36.699710 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":106,"threads_started":1}
I20260812 09:43:36.700348 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.693965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:45957 (local address 127.3.147.1:46269)
0812 09:43:36.695173 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.695177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.695190 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:43:36.695226 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.695229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.695889 (+ 660us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.696302 (+ 413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:43:36.696388 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.697038 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.698755 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.699295 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.700016 (+ 721us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.700076 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.700114 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.700207 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1153,"thread_start_us":103,"threads_started":1}
I20260812 09:43:36.700380 22201 raft_consensus.cc:2454] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d95759bf4a4641a2c745da8567abe6 in term 1.
I20260812 09:43:36.700457 22504 raft_consensus.cc:2454] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d95759bf4a4641a2c745da8567abe6 in term 1.
I20260812 09:43:36.700724 22278 leader_election.cc:302] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c989ce412a497c90399a4da816b383, 36d95759bf4a4641a2c745da8567abe6; no voters:
I20260812 09:43:36.700898 22575 raft_consensus.cc:2781] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:43:36.701052 22201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df0bdc9ee564669bc1a182585cfadf0" candidate_uuid: "09c989ce412a497c90399a4da816b383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7e98c354fe84854a31b3e7a00408844" is_pre_election: true
I20260812 09:43:36.701170 22575 raft_consensus.cc:686] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 1 LEADER]: Becoming Leader. State: Replica: 36d95759bf4a4641a2c745da8567abe6, State: Running, Role: LEADER
I20260812 09:43:36.701277 22201 raft_consensus.cc:2379] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09c989ce412a497c90399a4da816b383 in current term 1: Already voted for candidate 36d95759bf4a4641a2c745da8567abe6 in this term.
I20260812 09:43:36.701527 22575 consensus_queue.cc:237] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:36.701718 22429 leader_election.cc:302] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09c989ce412a497c90399a4da816b383; no voters: 36d95759bf4a4641a2c745da8567abe6, e7e98c354fe84854a31b3e7a00408844
I20260812 09:43:36.702010 22576 raft_consensus.cc:2726] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 09:43:36.704885 22035 catalog_manager.cc:5495] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2). New cstate: current_term: 1 leader_uuid: "36d95759bf4a4641a2c745da8567abe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } health_report { overall_health: UNKNOWN } } }
I20260812 09:43:36.717995 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.714716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36052 (local address 127.3.147.2:46085)
0812 09:43:36.714838 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:36.714841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:36.714856 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:43:36.714933 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:36.714935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:36.714984 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:36.715081 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:43:36.715310 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.715833 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.716711 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:36.716919 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:36.717000 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:36.717276 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:43:36.717676 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:43:36.717767 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:36.717818 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 09:43:36.717862 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 09:43:36.723965 22201 raft_consensus.cc:1261] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Refusing update from remote peer 36d95759bf4a4641a2c745da8567abe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:43:36.724007 22504 raft_consensus.cc:1261] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 1 FOLLOWER]: Refusing update from remote peer 36d95759bf4a4641a2c745da8567abe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:43:36.724661 22575 consensus_queue.cc:1035] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:43:36.724884 22584 consensus_queue.cc:1035] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:43:36.731797 22589 mvcc.cc:205] Tried to move back new op lower bound from 7317617937293799424 to 7317617937212538880. Current Snapshot: MvccSnapshot[applied={T|T < 7317617937293799424}]
I20260812 09:43:36.740485 22035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:43:36.740648 22035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:43:36.743111 22035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Sending DeleteTablet for 3 replicas of tablet 4df0bdc9ee564669bc1a182585cfadf0
I20260812 09:43:36.743821 22330 tablet_service.cc:1514] Processing DeleteTablet for tablet 4df0bdc9ee564669bc1a182585cfadf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:43:36 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:36044
I20260812 09:43:36.743925 22177 tablet_service.cc:1514] Processing DeleteTablet for tablet 4df0bdc9ee564669bc1a182585cfadf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:43:36 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:35894
I20260812 09:43:36.743988 22476 tablet_service.cc:1514] Processing DeleteTablet for tablet 4df0bdc9ee564669bc1a182585cfadf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:43:36 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:44122
I20260812 09:43:36.744410 22598 tablet_replica.cc:330] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: stopping tablet replica
I20260812 09:43:36.744508 22602 tablet_replica.cc:330] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: stopping tablet replica
I20260812 09:43:36.744618 22598 raft_consensus.cc:2229] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:43:36.744771 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 22080
I20260812 09:43:36.744803 22598 raft_consensus.cc:2258] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:43:36.747103 22602 raft_consensus.cc:2229] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:43:36.747449 22602 raft_consensus.cc:2258] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:43:36.749271 22602 ts_tablet_manager.cc:1905] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:43:36.752589 22602 ts_tablet_manager.cc:1918] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:43:36.752715 22602 log.cc:1196] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/4df0bdc9ee564669bc1a182585cfadf0
I20260812 09:43:36.753019 22601 tablet_replica.cc:330] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: stopping tablet replica
I20260812 09:43:36.753510 22602 ts_tablet_manager.cc:1939] T 4df0bdc9ee564669bc1a182585cfadf0 P 09c989ce412a497c90399a4da816b383: Deleting consensus metadata
I20260812 09:43:36.753489 22601 raft_consensus.cc:2229] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:43:36.753921 22601 raft_consensus.cc:2258] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:43:36.756070 22601 ts_tablet_manager.cc:1905] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:43:36.756347 22019 catalog_manager.cc:4841] TS 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343): tablet 4df0bdc9ee564669bc1a182585cfadf0 (table test-table [id=8bd52df96ad64e27a4ce77fdb457e044]) successfully deleted
I20260812 09:43:36.759836 22601 ts_tablet_manager.cc:1918] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:43:36.759953 22601 log.cc:1196] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/4df0bdc9ee564669bc1a182585cfadf0
I20260812 09:43:36.760402 22601 ts_tablet_manager.cc:1939] T 4df0bdc9ee564669bc1a182585cfadf0 P 36d95759bf4a4641a2c745da8567abe6: Deleting consensus metadata
I20260812 09:43:36.761713 22020 catalog_manager.cc:4841] TS 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085): tablet 4df0bdc9ee564669bc1a182585cfadf0 (table test-table [id=8bd52df96ad64e27a4ce77fdb457e044]) successfully deleted
W20260812 09:43:36.763720 22019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:46269 (error 108)
I20260812 09:43:36.764721 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 22257
I20260812 09:43:36.764997 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:36.764691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:46269 (local address 127.0.0.1:35906)
0812 09:43:36.764845 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:36.764911 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:46269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 09:43:36.766414 22019 catalog_manager.cc:4584] TS e7e98c354fe84854a31b3e7a00408844 (127.3.147.1:46269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4df0bdc9ee564669bc1a182585cfadf0: Network error: Client connection negotiation failed: client connection to 127.3.147.1:46269: connect: Connection refused (error 111)
I20260812 09:43:36.773507 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 22409
I20260812 09:43:36.782239 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 22000
I20260812 09:43:56.791544 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38965
--webserver_interface=127.3.147.62
--webserver_port=37671
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:38965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:43:56.905692 22604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:56.905961 22604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:56.905999 22604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:56.910156 22604 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 09:43:56.910205 22604 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 09:43:56.910220 22604 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 09:43:56.910234 22604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:43:56.910252 22604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:56.910295 22604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:56.910328 22604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:43:56.910347 22604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:43:56.915596 22604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:38965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:56.916718 22604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:56.917768 22604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:56.926554 22610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:56.926661 22604 server_base.cc:1034] running on GCE node
W20260812 09:43:56.926621 22612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:56.926887 22609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:56.927500 22604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:56.928716 22604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:56.929903 22604 hybrid_clock.cc:648] HybridClock initialized: now 1786527836929872 us; error 43 us; skew 500 ppm
Aug 12 09:43:56 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527836, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:56.935843 22604 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:43:56.936834 22604 webserver.cc:460] Webserver started at http://127.3.147.62:37671/ using document root <none> and password file <none>
I20260812 09:43:56.937374 22604 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:56.937436 22604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:56.941457 22604 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:43:56.943655 22620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:56.944943 22604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:43:56.945048 22604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "e35011e296454af99c25eba73ae2103b"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "6508ba3ecf5162d3f591099791cd9887"
server_key_iv: "b440fea6347ec89523eb3a37a9438881"
server_key_version: "encryptionkey@0"
I20260812 09:43:56.945609 22604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:56.965725 22604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:56.968713 22604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:56.968883 22604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:56.976243 22604 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:38965
I20260812 09:43:56.976604 22678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:38965 every 8 connection(s)
I20260812 09:43:56.977439 22604 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:43:56.979447 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 22604
I20260812 09:43:56.980041 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:46269
--local_ip_for_outbound_sockets=127.3.147.1
--tserver_master_addrs=127.3.147.62:38965
--webserver_port=33233
--webserver_interface=127.3.147.1
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:43:56.982939 22679 sys_catalog.cc:263] Verifying existing consensus state
I20260812 09:43:56.984923 22679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Bootstrap starting.
I20260812 09:43:57.004674 22679 log.cc:824] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:57.016527 22679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:43:57.016938 22679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Bootstrap complete.
I20260812 09:43:57.021423 22679 raft_consensus.cc:348] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:57.021836 22679 raft_consensus.cc:729] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35011e296454af99c25eba73ae2103b, State: Initialized, Role: FOLLOWER
I20260812 09:43:57.022365 22679 consensus_queue.cc:260] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:57.022486 22679 raft_consensus.cc:388] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:43:57.022580 22679 raft_consensus.cc:482] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:43:57.022707 22679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 2 FOLLOWER]: Advancing to term 3
I20260812 09:43:57.024785 22679 raft_consensus.cc:504] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:57.025178 22679 leader_election.cc:302] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e35011e296454af99c25eba73ae2103b; no voters:
I20260812 09:43:57.025590 22679 leader_election.cc:288] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 09:43:57.025668 22685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 3 FOLLOWER]: Leader election won for term 3
I20260812 09:43:57.025969 22685 raft_consensus.cc:686] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [term 3 LEADER]: Becoming Leader. State: Replica: e35011e296454af99c25eba73ae2103b, State: Running, Role: LEADER
I20260812 09:43:57.026281 22685 consensus_queue.cc:237] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } }
I20260812 09:43:57.026928 22679 sys_catalog.cc:564] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:43:57.027290 22685 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35011e296454af99c25eba73ae2103b. Latest consensus state: current_term: 3 leader_uuid: "e35011e296454af99c25eba73ae2103b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } } }
I20260812 09:43:57.027416 22685 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: This master's current role is: LEADER
I20260812 09:43:57.027803 22686 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e35011e296454af99c25eba73ae2103b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35011e296454af99c25eba73ae2103b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38965 } } }
I20260812 09:43:57.027987 22686 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b [sys.catalog]: This master's current role is: LEADER
I20260812 09:43:57.027822 22690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:43:57.033712 22690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:43:57.034626 22690 catalog_manager.cc:1259] Loaded cluster ID: 660e6563a012411eb50796cc9b8ec33a
I20260812 09:43:57.034690 22690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:43:57.037178 22690 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:43:57.037912 22690 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Loaded TSK: 0
I20260812 09:43:57.038009 22690 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e35011e296454af99c25eba73ae2103b: Loaded TSK: 1
F20260812 09:43:57.038717 22690 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdfe13bd9ad google::LogMessage::Fail() at ??:0
@ 0x7fdfe13c088a google::LogMessage::SendToLog() at ??:0
@ 0x7fdfe13bd4de google::LogMessage::Flush() at ??:0
@ 0x7fdfe13bea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdfe408118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fdfe4079f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fdfebd227bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fdfebcfc99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fdfebcfd178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fdfebd3edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fdfebd5d82b std::function<>::operator()() at ??:0
@ 0x7fdfebcfcb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fdfebcfdf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdfebcfa579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fdfebd3e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fdfe360f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdfe360fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fdfe36113f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fdfe3600512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdfe4c446db start_thread at ??:0
@ 0x7fdfdfebb61f clone at ??:0
W20260812 09:43:57.142055 22683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:57.142467 22683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:57.142590 22683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:57.147868 22683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:57.147936 22683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:57.148052 22683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:43:57.154237 22683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:46269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=33233
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:57.155424 22683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:57.156628 22683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:57.167069 22739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:57.167069 22737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:57.167155 22736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:57.167487 22683 server_base.cc:1034] running on GCE node
I20260812 09:43:57.167907 22683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:57.168617 22683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:57.169806 22683 hybrid_clock.cc:648] HybridClock initialized: now 1786527837169783 us; error 47 us; skew 500 ppm
Aug 12 09:43:57 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527837, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:57.173389 22683 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:43:57.174623 22683 webserver.cc:460] Webserver started at http://127.3.147.1:33233/ using document root <none> and password file <none>
I20260812 09:43:57.175384 22683 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:57.175493 22683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:57.180838 22683 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:43:57.183641 22747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:57.185228 22683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:43:57.185355 22683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "e7e98c354fe84854a31b3e7a00408844"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "639c9edb9e48c4b4c8da861153ffd740"
server_key_iv: "295ed69c2bb43cc2b5706102438e41c9"
server_key_version: "encryptionkey@0"
I20260812 09:43:57.186072 22683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:57.200897 22683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:57.204783 22683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:57.204993 22683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:57.205734 22683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:57.207132 22755 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 09:43:57.211264 22683 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 09:43:57.211340 22683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260812 09:43:57.211407 22683 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 09:43:57.213750 22683 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 09:43:57.213807 22683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 09:43:57.214135 22755 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Bootstrap starting.
I20260812 09:43:57.225247 22683 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:46269
I20260812 09:43:57.225471 22874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:46269 every 8 connection(s)
I20260812 09:43:57.226455 22683 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:43:57.228799 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 22683
I20260812 09:43:57.229411 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:46085
--local_ip_for_outbound_sockets=127.3.147.2
--tserver_master_addrs=127.3.147.62:38965
--webserver_port=46141
--webserver_interface=127.3.147.2
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:43:57.235570 22755 log.cc:824] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Log is configured to *not* fsync() on all Append() calls
I20260812 09:43:57.245905 22755 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 09:43:57.246413 22755 tablet_bootstrap.cc:492] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Bootstrap complete.
I20260812 09:43:57.248088 22755 ts_tablet_manager.cc:1397] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Time spent bootstrapping tablet: real 0.034s user 0.021s sys 0.009s
I20260812 09:43:57.251489 22755 raft_consensus.cc:348] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:57.252090 22755 raft_consensus.cc:729] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7e98c354fe84854a31b3e7a00408844, State: Initialized, Role: FOLLOWER
I20260812 09:43:57.252653 22755 consensus_queue.cc:260] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:57.253710 22755 ts_tablet_manager.cc:1428] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260812 09:43:57.281509 22878 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:43:57.228987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:41151)
0812 09:43:57.229524 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.229578 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:57.230518 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:57.280652 (+ 50134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":158,"threads_started":1}
W20260812 09:43:57.282183 22875 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:38965 (0 consecutive failures): Network error: Failed to ping master at 127.3.147.62:38965: Client connection negotiation failed: client connection to 127.3.147.62:38965: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 09:43:57.283209 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:57.282770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:60307)
0812 09:43:57.283046 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.283148 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260812 09:43:57.284011 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:57.283770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:43813)
0812 09:43:57.283893 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.283962 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 09:43:57.284312 22875 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 09:43:57.353241 22880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:57.353484 22880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:57.353525 22880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:57.357606 22880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:57.357657 22880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:57.357743 22880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:43:57.362986 22880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:46085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=46141
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:57.364001 22880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:57.365020 22880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:57.374890 22889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:57.374976 22880 server_base.cc:1034] running on GCE node
W20260812 09:43:57.374895 22887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:57.374927 22891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:57.375540 22880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:57.376233 22880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:57.377427 22880 hybrid_clock.cc:648] HybridClock initialized: now 1786527837377398 us; error 50 us; skew 500 ppm
Aug 12 09:43:57 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527837, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:57.380192 22880 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:43:57.381007 22880 webserver.cc:460] Webserver started at http://127.3.147.2:46141/ using document root <none> and password file <none>
I20260812 09:43:57.381574 22880 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:57.381672 22880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:57.385478 22880 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 09:43:57.387542 22899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:57.388756 22880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:43:57.388890 22880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "36d95759bf4a4641a2c745da8567abe6"
format_stamp: "Formatted at 2026-08-12 09:43:14 on dist-test-slave-cbbz"
server_key: "8995e397b4b9773811ed5d7b1916f86a"
server_key_iv: "9a4f20be0dbb19ae0126b447ea2913fa"
server_key_version: "encryptionkey@0"
I20260812 09:43:57.389462 22880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:57.429996 22880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:57.433220 22880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:57.433439 22880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:57.434051 22880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:57.434899 22880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:43:57.434947 22880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:57.435022 22880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:43:57.435067 22880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:57.444393 22880 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46085
I20260812 09:43:57.444427 23032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46085 every 8 connection(s)
I20260812 09:43:57.445410 22880 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:43:57.449476 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:57.447519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:51285)
0812 09:43:57.448018 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.448201 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
W20260812 09:43:57.449916 23033 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:38965 (0 consecutive failures): Network error: Failed to ping master at 127.3.147.62:38965: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
I20260812 09:43:57.450519 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:57.450294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:39435)
0812 09:43:57.450406 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.450458 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 09:43:57.451144 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:57.450928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:44059)
0812 09:43:57.451035 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.451080 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 09:43:57.451313 23033 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 09:43:57.451336 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 22880
I20260812 09:43:57.451741 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:41343
--local_ip_for_outbound_sockets=127.3.147.3
--tserver_master_addrs=127.3.147.62:38965
--webserver_port=45151
--webserver_interface=127.3.147.3
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:43:57.564076 23036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:43:57.564379 23036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:43:57.564455 23036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:43:57.568298 23036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:43:57.568372 23036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:43:57.568460 23036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:43:57.573412 23036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:41343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=45151
--tserver_master_addrs=127.3.147.62:38965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:43:57.574400 23036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:43:57.575425 23036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:43:57.585474 23046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:57.585448 23036 server_base.cc:1034] running on GCE node
W20260812 09:43:57.585469 23042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:43:57.585496 23043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:43:57.586187 23036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:43:57.586814 23036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:43:57.588020 23036 hybrid_clock.cc:648] HybridClock initialized: now 1786527837587984 us; error 46 us; skew 500 ppm
Aug 12 09:43:57 dist-test-slave-cbbz krb5kdc[21370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527837, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:43:57.590706 23036 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:43:57.591567 23036 webserver.cc:460] Webserver started at http://127.3.147.3:45151/ using document root <none> and password file <none>
I20260812 09:43:57.592243 23036 fs_manager.cc:359] Metadata directory not provided
I20260812 09:43:57.592350 23036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:43:57.596246 23036 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 09:43:57.598152 23055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:57.599399 23036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:43:57.599505 23036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "09c989ce412a497c90399a4da816b383"
format_stamp: "Formatted at 2026-08-12 09:43:15 on dist-test-slave-cbbz"
server_key: "b3fa0d649b5ccf90746f787afeda80a4"
server_key_iv: "852079846513fb407a96bbbcec1377c6"
server_key_version: "encryptionkey@0"
I20260812 09:43:57.600008 23036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:43:57.630838 23036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:43:57.633908 23036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:43:57.634068 23036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:43:57.634630 23036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:43:57.635421 23036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:43:57.635468 23036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:57.635506 23036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:43:57.635522 23036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:43:57.645048 23036 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:41343
I20260812 09:43:57.645100 23184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:41343 every 8 connection(s)
I20260812 09:43:57.646050 23036 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:43:57.648806 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 23036
I20260812 09:43:57.650315 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:57.648115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:51775)
0812 09:43:57.648676 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.648849 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":119,"threads_started":1}
W20260812 09:43:57.650794 23185 heartbeater.cc:644] Failed to heartbeat to 127.3.147.62:38965 (0 consecutive failures): Network error: Failed to ping master at 127.3.147.62:38965: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
I20260812 09:43:57.651373 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:57.651143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:33211)
0812 09:43:57.651266 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.651320 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 09:43:57.651988 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:57.651797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:33085)
0812 09:43:57.651895 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:57.651937 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 09:43:57.652235 23185 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 09:43:58.285441 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:58.284975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:42679)
0812 09:43:58.285249 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:58.285308 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
I20260812 09:43:58.452421 23190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:58.451887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:47019)
0812 09:43:58.452260 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:58.452318 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
I20260812 09:43:58.653154 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:58.652749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:59871)
0812 09:43:58.653000 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:58.653054 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260812 09:43:58.730075 23192 raft_consensus.cc:482] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:43:58.730350 23192 raft_consensus.cc:504] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:43:58.731602 23192 leader_election.cc:288] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085), 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343)
Aug 12 09:43:58 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527837, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.3@KRBTEST.COM
Aug 12 09:43:58 dist-test-slave-cbbz krb5kdc[21370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527837, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.2@KRBTEST.COM
I20260812 09:43:58.746436 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:58.732033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:33803 (local address 127.3.147.3:41343)
0812 09:43:58.732584 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:58.732609 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:58.732674 (+ 65us) server_negotiation.cc:407] Connection header received
0812 09:43:58.733408 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:58.733413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:58.733739 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:58.734033 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:58.735372 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:58.736452 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:58.737720 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:58.738203 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:58.738400 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:58.741387 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:58.741394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:58.741410 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:58.741445 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:58.743490 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:58.744056 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:58.744063 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:58.744069 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:58.744251 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:58.744729 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:58.744741 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:58.744742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:58.745202 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:58.745311 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:58.745976 (+ 665us) server_negotiation.cc:299] Negotiation successful
0812 09:43:58.746237 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":161,"threads_started":1}
I20260812 09:43:58.746459 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:58.731876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46085 (local address 127.3.147.1:50717)
0812 09:43:58.732524 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:58.732540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:58.732656 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:58.734432 (+ 1776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:58.734434 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:58.734703 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:58.735169 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:58.735184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:58.736630 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:58.736637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:58.737587 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:58.737598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:58.738364 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:58.738368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:58.738595 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:58.739394 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:58.739407 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:58.741553 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:58.744075 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:58.744079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:58.744081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:58.744354 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:58.744729 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:58.744732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:58.744734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:58.744845 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:58.745521 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:58.745523 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:58.745623 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 09:43:58.745846 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 09:43:58.746206 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":331,"spinlock_wait_cycles":13824,"thread_start_us":99,"threads_started":1}
I20260812 09:43:58.746515 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:58.731876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:41343 (local address 127.3.147.1:33803)
0812 09:43:58.732081 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:58.732099 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:43:58.732258 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:43:58.734115 (+ 1857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:43:58.734121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:43:58.734677 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:43:58.735168 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:58.735182 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:58.736599 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:58.736603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:58.737590 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:43:58.737599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:43:58.738349 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:43:58.738353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:43:58.738599 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:58.739334 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:43:58.739350 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:43:58.741121 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:43:58.743617 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:58.743624 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:58.743637 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:58.743895 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:58.744371 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:43:58.744374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:43:58.744376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:43:58.744589 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:43:58.745334 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:43:58.745341 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:43:58.745652 (+ 311us) client_negotiation.cc:769] Sending connection context
0812 09:43:58.745850 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 09:43:58.746312 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":312,"spinlock_wait_cycles":3712}
I20260812 09:43:58.747349 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:58.732185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:50717 (local address 127.3.147.2:46085)
0812 09:43:58.732639 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 09:43:58.732664 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 09:43:58.732708 (+ 44us) server_negotiation.cc:407] Connection header received
0812 09:43:58.733382 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:43:58.733388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:43:58.733735 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:43:58.734046 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:43:58.735363 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:58.736492 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:58.737729 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:43:58.738225 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:43:58.738411 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:43:58.741770 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:43:58.741780 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:43:58.741799 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:43:58.742106 (+ 307us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:43:58.743958 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:58.744472 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:58.744479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:58.744485 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:58.744628 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:43:58.744953 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:43:58.744955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:43:58.744957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:43:58.745404 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:43:58.745546 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 09:43:58.746012 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 09:43:58.746322 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":64,"threads_started":1}
W20260812 09:43:58.749523 22750 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
W20260812 09:43:58.749714 22748 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
I20260812 09:43:58.749821 22748 leader_election.cc:302] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7e98c354fe84854a31b3e7a00408844; no voters: 09c989ce412a497c90399a4da816b383, 36d95759bf4a4641a2c745da8567abe6
I20260812 09:43:58.750109 23192 raft_consensus.cc:2726] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:43:59.286676 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:59.286215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:33381)
0812 09:43:59.286515 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:59.286569 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
I20260812 09:43:59.453739 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:59.453250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:48211)
0812 09:43:59.453586 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:59.453645 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":137,"threads_started":1}
I20260812 09:43:59.654726 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:43:59.653940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:59617)
0812 09:43:59.654572 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0812 09:43:59.654638 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":471,"threads_started":1}
I20260812 09:44:00.288076 23201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:00.287616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:33559)
0812 09:44:00.287892 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:00.287964 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
I20260812 09:44:00.454902 23202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:00.454448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:33979)
0812 09:44:00.454725 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:00.454789 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":74,"threads_started":1}
I20260812 09:44:00.524611 23203 raft_consensus.cc:482] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:00.524757 23203 raft_consensus.cc:504] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:44:00.525246 23203 leader_election.cc:288] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085), 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343)
W20260812 09:44:00.526124 22750 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
W20260812 09:44:00.526214 22748 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
I20260812 09:44:00.526278 22748 leader_election.cc:302] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7e98c354fe84854a31b3e7a00408844; no voters: 09c989ce412a497c90399a4da816b383, 36d95759bf4a4641a2c745da8567abe6
I20260812 09:44:00.526427 23203 raft_consensus.cc:2726] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:44:00.655969 23204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:00.655446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:54375)
0812 09:44:00.655812 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:00.655868 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":88,"threads_started":1}
I20260812 09:44:01.289258 23205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:01.288824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:57473)
0812 09:44:01.289087 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:01.289143 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
I20260812 09:44:01.455945 23206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:01.455570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:49363)
0812 09:44:01.455801 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:01.455857 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":77,"threads_started":1}
I20260812 09:44:01.657322 23207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:01.656706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:59847)
0812 09:44:01.657163 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:01.657218 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":263,"threads_started":1}
I20260812 09:44:02.032833 23209 raft_consensus.cc:482] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:02.032970 23209 raft_consensus.cc:504] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:44:02.033574 23209 leader_election.cc:288] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085), 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343)
W20260812 09:44:02.034377 22748 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
W20260812 09:44:02.034473 22750 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
I20260812 09:44:02.034605 22750 leader_election.cc:302] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7e98c354fe84854a31b3e7a00408844; no voters: 09c989ce412a497c90399a4da816b383, 36d95759bf4a4641a2c745da8567abe6
I20260812 09:44:02.034783 23209 raft_consensus.cc:2726] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:44:02.290917 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:02.290372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:54091)
0812 09:44:02.290764 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:02.290816 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":93,"threads_started":1}
I20260812 09:44:02.457141 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:02.456699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:49107)
0812 09:44:02.456978 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:02.457036 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":102,"threads_started":1}
I20260812 09:44:02.658432 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:02.658038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:42219)
0812 09:44:02.658284 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:02.658340 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260812 09:44:03.292106 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:03.291667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:38067)
0812 09:44:03.291950 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:03.292005 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":101,"threads_started":1}
I20260812 09:44:03.458379 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:03.457976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:52917)
0812 09:44:03.458215 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:03.458270 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
I20260812 09:44:03.660260 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:03.659204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:47175)
0812 09:44:03.660017 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:03.660072 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":649,"threads_started":1}
I20260812 09:44:04.293418 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:04.292983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:36959)
0812 09:44:04.293248 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:04.293316 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
I20260812 09:44:04.459698 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:04.459123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:52185)
0812 09:44:04.459520 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:04.459584 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":89,"threads_started":1}
I20260812 09:44:04.661633 23219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:04.661114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:60949)
0812 09:44:04.661466 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:04.661530 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":113,"threads_started":1}
I20260812 09:44:05.294651 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:05.294218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:38111)
0812 09:44:05.294492 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:05.294547 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260812 09:44:05.461040 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:05.460612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:38639)
0812 09:44:05.460887 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:05.460945 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
I20260812 09:44:05.662912 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:05.662459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:40531)
0812 09:44:05.662757 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:05.662814 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":120,"threads_started":1}
I20260812 09:44:06.295877 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:06.295426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:38671)
0812 09:44:06.295700 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:06.295762 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":81,"threads_started":1}
I20260812 09:44:06.462235 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:06.461753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:50113)
0812 09:44:06.462077 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:06.462134 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":132,"threads_started":1}
I20260812 09:44:06.563019 23227 raft_consensus.cc:482] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:06.563228 23227 raft_consensus.cc:504] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:44:06.563722 23227 leader_election.cc:288] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085), 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343)
W20260812 09:44:06.564590 22748 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
W20260812 09:44:06.564702 22750 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
I20260812 09:44:06.564746 22748 leader_election.cc:302] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7e98c354fe84854a31b3e7a00408844; no voters: 09c989ce412a497c90399a4da816b383, 36d95759bf4a4641a2c745da8567abe6
I20260812 09:44:06.565001 23227 raft_consensus.cc:2726] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:44:06.664319 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:06.663802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:43923)
0812 09:44:06.664082 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:06.664215 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
I20260812 09:44:07.297211 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:07.296742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:56333)
0812 09:44:07.297001 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:07.297075 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260812 09:44:07.463563 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:07.463040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:38819)
0812 09:44:07.463389 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:07.463454 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":118,"threads_started":1}
I20260812 09:44:07.665498 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:07.665080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:34433)
0812 09:44:07.665347 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:07.665401 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
I20260812 09:44:08.298329 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:08.297893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:58727)
0812 09:44:08.298178 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:08.298230 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":91,"threads_started":1}
I20260812 09:44:08.358279 23233 raft_consensus.cc:482] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:08.358449 23233 raft_consensus.cc:504] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d95759bf4a4641a2c745da8567abe6" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46085 } } peers { permanent_uuid: "09c989ce412a497c90399a4da816b383" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 41343 } } peers { permanent_uuid: "e7e98c354fe84854a31b3e7a00408844" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 46269 } }
I20260812 09:44:08.358925 23233 leader_election.cc:288] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085), 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343)
W20260812 09:44:08.359714 22748 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c989ce412a497c90399a4da816b383 (127.3.147.3:41343): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
W20260812 09:44:08.359802 22750 leader_election.cc:341] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d95759bf4a4641a2c745da8567abe6 (127.3.147.2:46085): Not found: Tablet not found: 4df0bdc9ee564669bc1a182585cfadf0
I20260812 09:44:08.359937 22750 leader_election.cc:302] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7e98c354fe84854a31b3e7a00408844; no voters: 09c989ce412a497c90399a4da816b383, 36d95759bf4a4641a2c745da8567abe6
I20260812 09:44:08.360116 23233 raft_consensus.cc:2726] T 4df0bdc9ee564669bc1a182585cfadf0 P e7e98c354fe84854a31b3e7a00408844 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 09:44:08.464808 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:08.464394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:33267)
0812 09:44:08.464634 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:08.464692 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260812 09:44:08.666698 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:08.666225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:58749)
0812 09:44:08.666525 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:08.666591 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":115,"threads_started":1}
I20260812 09:44:09.299567 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:09.299078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:47411)
0812 09:44:09.299391 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:09.299451 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":79,"threads_started":1}
I20260812 09:44:09.466099 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:09.465595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:51959)
0812 09:44:09.465927 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:09.465994 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":102,"threads_started":1}
I20260812 09:44:09.667909 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:09.667480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:56965)
0812 09:44:09.667759 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:09.667813 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":92,"threads_started":1}
I20260812 09:44:10.300889 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:10.300401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:60071)
0812 09:44:10.300682 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:10.300743 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260812 09:44:10.467415 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:10.466955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:43667)
0812 09:44:10.467262 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:10.467317 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":113,"threads_started":1}
I20260812 09:44:10.669014 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:10.668614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:39167)
0812 09:44:10.668864 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:10.668918 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
I20260812 09:44:11.302145 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:11.301678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:37091)
0812 09:44:11.301970 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:11.302033 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":85,"threads_started":1}
I20260812 09:44:11.468736 23245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:11.468304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:47877)
0812 09:44:11.468584 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:11.468638 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":76,"threads_started":1}
I20260812 09:44:11.670269 23246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:11.669822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:33413)
0812 09:44:11.670108 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:11.670170 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":81,"threads_started":1}
I20260812 09:44:12.303666 23247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:12.303199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.1:55361)
0812 09:44:12.303514 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:12.303566 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":145,"threads_started":1}
I20260812 09:44:12.470088 23248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:12.469626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.2:48323)
0812 09:44:12.469899 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:12.469962 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
I20260812 09:44:12.653834 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 22683
I20260812 09:44:12.663957 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 22880
I20260812 09:44:12.671733 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:12.671308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:38965 (local address 127.3.147.3:41763)
0812 09:44:12.671582 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:12.671638 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.62:38965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
I20260812 09:44:12.671969 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 23036
2026-08-12T09:44:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59569 ms)
[----------] 30 tests from SecurityITest (248992 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:12 dist-test-slave-cbbz krb5kdc[23252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:12 dist-test-slave-cbbz krb5kdc[23252](info): set up 2 sockets
Aug 12 09:44:12 dist-test-slave-cbbz krb5kdc[23252](info): commencing operation
krb5kdc: starting...
W20260812 09:44:13.900589 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:13 dist-test-slave-cbbz krb5kdc[23252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:14.027275 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:36489
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:36489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:14.137995 23267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:14.138301 23267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:14.138408 23267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:14.142601 23267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:14.142678 23267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:14.142719 23267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:14.142756 23267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:14.142791 23267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:14.147828 23267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:36489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:36489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:14.148959 23267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:14.150044 23267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:14.158636 23274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:14.158684 23276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:14.158636 23272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:14.159123 23267 server_base.cc:1034] running on GCE node
I20260812 09:44:14.159698 23267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:14.160867 23267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:14.162060 23267 hybrid_clock.cc:648] HybridClock initialized: now 1786527854162057 us; error 37 us; skew 500 ppm
Aug 12 09:44:14 dist-test-slave-cbbz krb5kdc[23252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:14.164808 23267 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:14.165647 23267 webserver.cc:460] Webserver started at http://127.3.147.62:37715/ using document root <none> and password file <none>
I20260812 09:44:14.166199 23267 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:14.166299 23267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:14.166540 23267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:14.168629 23267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "0043979aff174a97b53d40fedd492f49"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "05ba35f3f7cc968d01b1eab8c31d858c"
server_key_iv: "87695bd4732983c190c832854139e6cb"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.169216 23267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "0043979aff174a97b53d40fedd492f49"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "05ba35f3f7cc968d01b1eab8c31d858c"
server_key_iv: "87695bd4732983c190c832854139e6cb"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.172959 23267 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:44:14.175411 23286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.176678 23267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:44:14.176815 23267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "0043979aff174a97b53d40fedd492f49"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "05ba35f3f7cc968d01b1eab8c31d858c"
server_key_iv: "87695bd4732983c190c832854139e6cb"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.176929 23267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:14.194895 23267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:14.198005 23267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:14.198160 23267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:14.204954 23267 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:36489
I20260812 09:44:14.205052 23342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:36489 every 8 connection(s)
I20260812 09:44:14.205925 23267 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:14.208693 23343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:14.213876 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 23267
I20260812 09:44:14.213979 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:14.214136 23343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49: Bootstrap starting.
I20260812 09:44:14.214339 3660 external_mini_cluster.cc:1428] Setting key 2f901fd9dde6bca72b9bc092e937afa6
I20260812 09:44:14.216902 23343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:14.217864 23343 log.cc:824] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:14.220157 23343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49: No bootstrap required, opened a new log
I20260812 09:44:14.224097 23343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0043979aff174a97b53d40fedd492f49" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 36489 } }
Aug 12 09:44:14 dist-test-slave-cbbz krb5kdc[23252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:14.224413 23343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:14.224501 23343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0043979aff174a97b53d40fedd492f49, State: Initialized, Role: FOLLOWER
I20260812 09:44:14.225019 23343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0043979aff174a97b53d40fedd492f49" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 36489 } }
I20260812 09:44:14.225167 23343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:14.225211 23343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:14.225274 23343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:14.226166 23343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0043979aff174a97b53d40fedd492f49" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 36489 } }
I20260812 09:44:14.226507 23343 leader_election.cc:302] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0043979aff174a97b53d40fedd492f49; no voters:
I20260812 09:44:14.226815 23343 leader_election.cc:288] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:14.226958 23348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:14.227191 23348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [term 1 LEADER]: Becoming Leader. State: Replica: 0043979aff174a97b53d40fedd492f49, State: Running, Role: LEADER
I20260812 09:44:14.227572 23348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0043979aff174a97b53d40fedd492f49" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 36489 } }
I20260812 09:44:14.228075 23343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:14.230278 23349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0043979aff174a97b53d40fedd492f49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0043979aff174a97b53d40fedd492f49" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 36489 } } }
I20260812 09:44:14.230829 23349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:14.231588 23353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0043979aff174a97b53d40fedd492f49. Latest consensus state: current_term: 1 leader_uuid: "0043979aff174a97b53d40fedd492f49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0043979aff174a97b53d40fedd492f49" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 36489 } } }
I20260812 09:44:14.231922 23353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:14.231931 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.215735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.3.147.62:36489)
0812 09:44:14.216193 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:14.216213 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:14.216258 (+ 45us) server_negotiation.cc:407] Connection header received
0812 09:44:14.217179 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:14.217204 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:14.217643 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:14.218055 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:14.219047 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.220522 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.221447 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.222183 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.222341 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:14.225097 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:14.225103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:14.225121 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:14.225273 (+ 152us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:14.227410 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.227959 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.227965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.227971 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.228110 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.228610 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.228614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.228616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.229202 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:14.230187 (+ 985us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:14.230661 (+ 474us) server_negotiation.cc:299] Negotiation successful
0812 09:44:14.230953 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":111,"threads_started":1}
W20260812 09:44:14.233433 23367 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:14.233580 23367 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:14.234395 23368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:14.235340 23368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:14.240972 23368 catalog_manager.cc:1347] Generated new cluster ID: 10a8c0256b894a278544a74ef0354854
I20260812 09:44:14.241048 23368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:14.247968 23368 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:14.249194 23368 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:14.256664 23368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49: Generated new TSK 0
I20260812 09:44:14.257468 23368 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:44:14.294876 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:36489
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:14.408823 23376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:14.409061 23376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:14.409116 23376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:14.413154 23376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:14.413203 23376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:14.413292 23376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:44:14.418455 23376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:36489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:14.419477 23376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:14.420495 23376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:14.430003 23385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:14.430018 23382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:14.430135 23376 server_base.cc:1034] running on GCE node
W20260812 09:44:14.430325 23383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:14.430714 23376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:14.431372 23376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:14.432554 23376 hybrid_clock.cc:648] HybridClock initialized: now 1786527854432544 us; error 29 us; skew 500 ppm
Aug 12 09:44:14 dist-test-slave-cbbz krb5kdc[23252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:14.435184 23376 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:44:14.435990 23376 webserver.cc:460] Webserver started at http://127.3.147.1:42565/ using document root <none> and password file <none>
I20260812 09:44:14.436609 23376 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:14.436705 23376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:14.436935 23376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:14.438902 23376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "bd89a1e671274fb4ac30bfea61801370"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "6ef1207353a8dd031c0604264bc831c3"
server_key_iv: "b02923994473c6cbaf37b8ef8adf96fd"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.439479 23376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "bd89a1e671274fb4ac30bfea61801370"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "6ef1207353a8dd031c0604264bc831c3"
server_key_iv: "b02923994473c6cbaf37b8ef8adf96fd"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.443377 23376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 09:44:14.445772 23393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.446863 23376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:44:14.446991 23376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "bd89a1e671274fb4ac30bfea61801370"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "6ef1207353a8dd031c0604264bc831c3"
server_key_iv: "b02923994473c6cbaf37b8ef8adf96fd"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.447095 23376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:14.471475 23376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:14.474361 23376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:14.474565 23376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:14.475147 23376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:14.475968 23376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:14.476042 23376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.476111 23376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:14.476181 23376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.485689 23376 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:43613
I20260812 09:44:14.485720 23525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:43613 every 8 connection(s)
I20260812 09:44:14.486593 23376 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:44:14.491930 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 23376
I20260812 09:44:14.492074 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:44:14.492403 3660 external_mini_cluster.cc:1428] Setting key 44db0a597982f729362c2e0c61e21be9
Aug 12 09:44:14 dist-test-slave-cbbz krb5kdc[23252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:14.500627 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.488540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:35843 (local address 127.3.147.62:36489)
0812 09:44:14.488686 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:14.488690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:14.489544 (+ 854us) server_negotiation.cc:407] Connection header received
0812 09:44:14.490489 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:14.490493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:14.490547 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:14.490663 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:14.491810 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.492561 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.493906 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.494175 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.494283 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:14.496971 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:14.496978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:14.496985 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:14.497018 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:14.498445 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.499041 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.499046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.499048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.499130 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.499465 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.499470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.499472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.499647 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:14.499751 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:14.500311 (+ 560us) server_negotiation.cc:299] Negotiation successful
0812 09:44:14.500453 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 09:44:14.501520 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.488925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:36489 (local address 127.3.147.1:35843)
0812 09:44:14.489396 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:14.489437 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:14.490267 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:14.490839 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:14.490845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:14.491196 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:14.491667 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.491681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.492702 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.492706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:14.493787 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.493798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.494337 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.494342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:14.494511 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:14.495138 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:14.495154 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:14.496796 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:14.498562 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:14.498568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:14.498582 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:14.498942 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:14.499221 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:14.499226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:14.499228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:14.499370 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:14.499750 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:14.499766 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:14.500023 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 09:44:14.500264 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 09:44:14.500598 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:44:14.502954 23529 heartbeater.cc:347] Connected to a master server at 127.3.147.62:36489
I20260812 09:44:14.503234 23529 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:14.503734 23529 heartbeater.cc:511] Master 127.3.147.62:36489 requested a full tablet report, sending...
I20260812 09:44:14.505317 23305 ts_manager.cc:194] Registered new tserver with Master: bd89a1e671274fb4ac30bfea61801370 (127.3.147.1:43613)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:44:14.506685 23305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:35843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:44:14.523571 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:36489
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:14.635172 23534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:14.635464 23534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:14.635550 23534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:14.640112 23534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:14.640214 23534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:14.640339 23534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:44:14.648240 23534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:36489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:14.649240 23534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:14.650239 23534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:14.660063 23545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:14.660223 23539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:14.660290 23534 server_base.cc:1034] running on GCE node
W20260812 09:44:14.660063 23540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:14.660843 23534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:14.661454 23534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:14.662636 23534 hybrid_clock.cc:648] HybridClock initialized: now 1786527854662623 us; error 27 us; skew 500 ppm
Aug 12 09:44:14 dist-test-slave-cbbz krb5kdc[23252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:14.665309 23534 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:44:14.666112 23534 webserver.cc:460] Webserver started at http://127.3.147.2:45999/ using document root <none> and password file <none>
I20260812 09:44:14.666662 23534 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:14.666754 23534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:14.666980 23534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:14.669024 23534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "157c462b5f3a4165b8e2fb67c519a184"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "88b5fc89f89f4088bc4f56e8f999fb8c"
server_key_iv: "9cef53d230d2c6dec088685babb78d89"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.669592 23534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "157c462b5f3a4165b8e2fb67c519a184"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "88b5fc89f89f4088bc4f56e8f999fb8c"
server_key_iv: "9cef53d230d2c6dec088685babb78d89"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.673244 23534 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260812 09:44:14.675537 23552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.676735 23534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:14.676827 23534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "157c462b5f3a4165b8e2fb67c519a184"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "88b5fc89f89f4088bc4f56e8f999fb8c"
server_key_iv: "9cef53d230d2c6dec088685babb78d89"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.676906 23534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:14.701589 23534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:14.704388 23534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:14.704538 23534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:14.705075 23534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:14.705873 23534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:14.705919 23534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.705960 23534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:14.705974 23534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.715200 23534 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:46757
I20260812 09:44:14.715240 23687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:46757 every 8 connection(s)
I20260812 09:44:14.716105 23534 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:44:14.720541 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 23534
I20260812 09:44:14.720717 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:44:14.721078 3660 external_mini_cluster.cc:1428] Setting key a29fd6a3d2b56aa296657cc2d3b3d1a6
Aug 12 09:44:14 dist-test-slave-cbbz krb5kdc[23252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:14.731196 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.717969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:52841 (local address 127.3.147.62:36489)
0812 09:44:14.718131 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:14.718137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:14.719146 (+ 1009us) server_negotiation.cc:407] Connection header received
0812 09:44:14.720005 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:14.720008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:14.720054 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:14.720199 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:14.721589 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.722134 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.723378 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.723767 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.723885 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:14.726971 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:14.726975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:14.726977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:14.727011 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:14.728618 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.729168 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.729172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.729174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.729289 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.729723 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.729730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.729759 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.730080 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:14.730193 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:14.730838 (+ 645us) server_negotiation.cc:299] Negotiation successful
0812 09:44:14.730970 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:44:14.732095 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.718438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:36489 (local address 127.3.147.2:52841)
0812 09:44:14.718968 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:14.719025 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:14.719797 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:14.720372 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:14.720378 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:14.720856 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:14.721402 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.721417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.722260 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.722264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:14.723232 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.723241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.723931 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.723934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:14.724082 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:14.725015 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:14.725043 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:14.726801 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:14.728743 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:14.728749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:14.728764 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:14.729037 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:14.729406 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:14.729408 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:14.729410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:14.729595 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:14.730289 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:14.730298 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:14.730564 (+ 266us) client_negotiation.cc:769] Sending connection context
0812 09:44:14.730746 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 09:44:14.730997 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":115,"threads_started":1}
I20260812 09:44:14.733400 23688 heartbeater.cc:347] Connected to a master server at 127.3.147.62:36489
I20260812 09:44:14.733673 23688 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:14.734210 23688 heartbeater.cc:511] Master 127.3.147.62:36489 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:44:14.735464 23305 ts_manager.cc:194] Registered new tserver with Master: 157c462b5f3a4165b8e2fb67c519a184 (127.3.147.2:46757)
I20260812 09:44:14.736357 23305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:52841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:44:14.751704 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:36489
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:14.864905 23693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:14.865181 23693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:14.865260 23693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:14.869418 23693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:14.869494 23693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:14.869598 23693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:44:14.874799 23693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:36489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:14.875831 23693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:14.876895 23693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:14.886889 23701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:14.886894 23700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:14.886945 23693 server_base.cc:1034] running on GCE node
W20260812 09:44:14.886889 23703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:14.887631 23693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:14.888304 23693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:14.889487 23693 hybrid_clock.cc:648] HybridClock initialized: now 1786527854889474 us; error 28 us; skew 500 ppm
Aug 12 09:44:14 dist-test-slave-cbbz krb5kdc[23252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:14.892257 23693 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:44:14.893081 23693 webserver.cc:460] Webserver started at http://127.3.147.3:37543/ using document root <none> and password file <none>
I20260812 09:44:14.893641 23693 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:14.893731 23693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:14.893992 23693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:14.896112 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "af3fc285a9c04ce7b7b788196de36c46"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "409329b4f45893ef19f903582407a29a"
server_key_iv: "84a78dd1c34bb201ca3741c935e8c61f"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.896724 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "af3fc285a9c04ce7b7b788196de36c46"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "409329b4f45893ef19f903582407a29a"
server_key_iv: "84a78dd1c34bb201ca3741c935e8c61f"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.900431 23693 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:14.902801 23710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.903977 23693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:14.904129 23693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "af3fc285a9c04ce7b7b788196de36c46"
format_stamp: "Formatted at 2026-08-12 09:44:14 on dist-test-slave-cbbz"
server_key: "409329b4f45893ef19f903582407a29a"
server_key_iv: "84a78dd1c34bb201ca3741c935e8c61f"
server_key_version: "encryptionkey@0"
I20260812 09:44:14.904290 23693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:14.935956 23693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:14.939268 23693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:14.939489 23693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:14.940120 23693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:14.940994 23693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:14.941067 23693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.941148 23693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:14.941200 23693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:14.951000 23693 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:42513
I20260812 09:44:14.951045 23835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:42513 every 8 connection(s)
I20260812 09:44:14.951905 23693 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:44:14.958595 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 23693
I20260812 09:44:14.958717 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:44:14.959064 3660 external_mini_cluster.cc:1428] Setting key 6ab9039ede72b9c533d329720e2d88b0
Aug 12 09:44:14 dist-test-slave-cbbz krb5kdc[23252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:14.965582 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.953845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:41985 (local address 127.3.147.62:36489)
0812 09:44:14.954021 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:14.954025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:14.954875 (+ 850us) server_negotiation.cc:407] Connection header received
0812 09:44:14.955693 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:14.955696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:14.955749 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:14.955837 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:14.957030 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.957574 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.958905 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.959192 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.959288 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:14.962226 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:14.962229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:14.962231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:14.962256 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:14.963712 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.964243 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.964247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.964248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.964333 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.964650 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.964653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.964654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.964836 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:14.964918 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:14.965364 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 09:44:14.965465 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 09:44:14.966355 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.954243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:36489 (local address 127.3.147.3:41985)
0812 09:44:14.954710 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:14.954753 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:14.955497 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:14.955996 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:14.956002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:14.956427 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:14.956890 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.956904 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.957691 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.957694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:14.958780 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.958793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.959372 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.959376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:14.959555 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:14.960284 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:14.960305 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:14.962079 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:14.963811 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:14.963815 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:14.963825 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:14.964078 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:14.964424 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:14.964427 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:14.964428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:14.964555 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:14.964927 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:14.964932 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:14.965157 (+ 225us) client_negotiation.cc:769] Sending connection context
0812 09:44:14.965314 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 09:44:14.965549 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":131,"threads_started":1}
I20260812 09:44:14.967538 23836 heartbeater.cc:347] Connected to a master server at 127.3.147.62:36489
I20260812 09:44:14.967770 23836 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:14.968340 23836 heartbeater.cc:511] Master 127.3.147.62:36489 requested a full tablet report, sending...
I20260812 09:44:14.969389 23305 ts_manager.cc:194] Registered new tserver with Master: af3fc285a9c04ce7b7b788196de36c46 (127.3.147.3:42513)
I20260812 09:44:14.969954 23305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:41985
I20260812 09:44:14.973209 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:44:14.982496 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.975264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.3.147.62:36489)
0812 09:44:14.975378 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:14.975381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:14.975553 (+ 172us) server_negotiation.cc:407] Connection header received
0812 09:44:14.975656 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:14.975658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:14.975694 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:14.975754 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:14.976443 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.976952 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.977740 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:14.977939 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:14.978025 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:14.979254 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:14.979257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:14.979259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:14.979281 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:14.980805 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.981209 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.981212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.981213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.981295 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:14.981596 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:14.981600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:14.981602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:14.981835 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:14.981981 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:14.982210 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 09:44:14.982333 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 09:44:14.985311 23305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:44:14.988457 23305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:44:15.003648 23849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.997166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.3.147.3:42513)
0812 09:44:14.997739 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:14.997754 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:14.997780 (+ 26us) server_negotiation.cc:407] Connection header received
0812 09:44:14.998015 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:14.998024 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:14.998253 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:14.998865 (+ 612us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:14.999042 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.000229 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.002043 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.002847 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.002961 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.003118 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.003347 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.003445 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":195,"threads_started":1}
I20260812 09:44:15.003662 23848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.996958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:42513 (local address 127.0.0.1:33050)
0812 09:44:14.997372 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:14.997427 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:14.997572 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:14.998455 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:14.998461 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:14.998488 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:14.998675 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.998683 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.000385 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.000388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.001907 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.001918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.002990 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.003005 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.003125 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.003167 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 09:44:15.003312 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 09:44:15.003436 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":51,"thread_start_us":85,"threads_started":1}
I20260812 09:44:15.003834 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.997688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46757 (local address 127.0.0.1:57322)
0812 09:44:14.998931 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:14.998944 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:14.999085 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:14.999587 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:14.999589 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:14.999602 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:14.999671 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.999675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.001051 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.001067 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.002159 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.002171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.003446 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.003450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.003544 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.003573 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 09:44:15.003635 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 09:44:15.003699 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"mutex_wait_us":28,"thread_start_us":94,"threads_started":1}
I20260812 09:44:15.004297 23857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.998285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57322 (local address 127.3.147.2:46757)
0812 09:44:14.998792 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:14.998801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:14.999024 (+ 223us) server_negotiation.cc:407] Connection header received
0812 09:44:14.999245 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:14.999251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:14.999441 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:14.999608 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:14.999776 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.000875 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.002294 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.002853 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.003010 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.003156 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.004000 (+ 844us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.004103 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":134,"threads_started":1}
I20260812 09:44:15.004623 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.997933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60824 (local address 127.3.147.1:43613)
0812 09:44:14.998417 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:14.998436 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:14.998459 (+ 23us) server_negotiation.cc:407] Connection header received
0812 09:44:14.998549 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:14.998554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:14.998785 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:14.998939 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:14.999796 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.000794 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.002333 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.003114 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.003250 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.003374 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.003619 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.004283 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":74,"threads_started":1}
I20260812 09:44:15.004932 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:14.997364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:43613 (local address 127.0.0.1:60824)
0812 09:44:14.997998 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:14.998013 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:14.998119 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:14.998937 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:14.998943 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:14.998966 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:14.999069 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:14.999074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.000949 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.000954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.002203 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.002211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.003249 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.003252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.003342 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.003364 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 09:44:15.003442 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 09:44:15.003509 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":86,"thread_start_us":92,"threads_started":1}
I20260812 09:44:15.006843 23611 tablet_service.cc:1467] Processing CreateTablet for tablet 9321e65aaf594043b027dafea25c5f04 (DEFAULT_TABLE table=test-table [id=935a3c158a2b40b59f5a8e84e66f4e31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:15.006935 23762 tablet_service.cc:1467] Processing CreateTablet for tablet 9321e65aaf594043b027dafea25c5f04 (DEFAULT_TABLE table=test-table [id=935a3c158a2b40b59f5a8e84e66f4e31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:15.007493 23448 tablet_service.cc:1467] Processing CreateTablet for tablet 9321e65aaf594043b027dafea25c5f04 (DEFAULT_TABLE table=test-table [id=935a3c158a2b40b59f5a8e84e66f4e31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:15.007953 23611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9321e65aaf594043b027dafea25c5f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:15.008083 23762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9321e65aaf594043b027dafea25c5f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:15.008491 23448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9321e65aaf594043b027dafea25c5f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:15.020763 23858 tablet_bootstrap.cc:492] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Bootstrap starting.
I20260812 09:44:15.020812 23859 tablet_bootstrap.cc:492] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Bootstrap starting.
I20260812 09:44:15.022630 23860 tablet_bootstrap.cc:492] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370: Bootstrap starting.
I20260812 09:44:15.023703 23859 tablet_bootstrap.cc:654] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:15.023803 23858 tablet_bootstrap.cc:654] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:15.024550 23860 tablet_bootstrap.cc:654] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:15.024698 23858 log.cc:824] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:15.024698 23859 log.cc:824] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:15.025501 23860 log.cc:824] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:15.026813 23858 tablet_bootstrap.cc:492] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: No bootstrap required, opened a new log
I20260812 09:44:15.027002 23858 ts_tablet_manager.cc:1397] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 09:44:15.027055 23859 tablet_bootstrap.cc:492] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: No bootstrap required, opened a new log
I20260812 09:44:15.027237 23859 ts_tablet_manager.cc:1397] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 09:44:15.027535 23860 tablet_bootstrap.cc:492] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370: No bootstrap required, opened a new log
I20260812 09:44:15.027741 23860 ts_tablet_manager.cc:1397] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 09:44:15.031011 23859 raft_consensus.cc:348] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.031011 23858 raft_consensus.cc:348] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.031302 23858 raft_consensus.cc:374] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:15.031302 23859 raft_consensus.cc:374] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:15.031364 23858 raft_consensus.cc:729] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 157c462b5f3a4165b8e2fb67c519a184, State: Initialized, Role: FOLLOWER
I20260812 09:44:15.031373 23859 raft_consensus.cc:729] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3fc285a9c04ce7b7b788196de36c46, State: Initialized, Role: FOLLOWER
I20260812 09:44:15.031829 23858 consensus_queue.cc:260] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.031829 23859 consensus_queue.cc:260] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.031903 23860 raft_consensus.cc:348] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.032225 23860 raft_consensus.cc:374] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:15.032307 23860 raft_consensus.cc:729] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd89a1e671274fb4ac30bfea61801370, State: Initialized, Role: FOLLOWER
I20260812 09:44:15.032797 23860 consensus_queue.cc:260] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.032884 23859 ts_tablet_manager.cc:1428] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:44:15.032884 23858 ts_tablet_manager.cc:1428] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 09:44:15.032943 23836 heartbeater.cc:503] Master 127.3.147.62:36489 was elected leader, sending a full tablet report...
I20260812 09:44:15.033514 23529 heartbeater.cc:503] Master 127.3.147.62:36489 was elected leader, sending a full tablet report...
I20260812 09:44:15.033576 23688 heartbeater.cc:503] Master 127.3.147.62:36489 was elected leader, sending a full tablet report...
I20260812 09:44:15.033812 23860 ts_tablet_manager.cc:1428] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 09:44:15.152477 23865 raft_consensus.cc:482] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:15.152729 23865 raft_consensus.cc:504] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.153960 23865 leader_election.cc:288] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 157c462b5f3a4165b8e2fb67c519a184 (127.3.147.2:46757), bd89a1e671274fb4ac30bfea61801370 (127.3.147.1:43613)
I20260812 09:44:15.158324 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.154286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:43613 (local address 127.3.147.3:47563)
0812 09:44:15.154491 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:15.154507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:15.154605 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:15.155040 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:15.155044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:15.155065 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:15.155219 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.155244 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.156180 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.156184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.157137 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.157146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.157838 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.157841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.157947 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.157965 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 09:44:15.158063 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 09:44:15.158155 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":32}
I20260812 09:44:15.158324 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.154406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:47563 (local address 127.3.147.1:43613)
0812 09:44:15.154538 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:15.154544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:15.154559 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:44:15.154788 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:15.154792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:15.154848 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:15.154916 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:15.155386 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.156044 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.157256 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.157714 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.157831 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.157894 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.158080 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.158177 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 09:44:15.158361 23857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.154489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:33711 (local address 127.3.147.2:46757)
0812 09:44:15.154621 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:15.154625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:15.155110 (+ 485us) server_negotiation.cc:407] Connection header received
0812 09:44:15.155271 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:15.155275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:15.155347 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:15.155427 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:15.155729 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.156250 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.157347 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.157765 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.157879 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.157937 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.158155 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.158233 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:15.158572 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.154286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:46757 (local address 127.3.147.3:33711)
0812 09:44:15.154983 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:15.154995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:15.155133 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:15.155459 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:15.155463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:15.155481 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:15.155615 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.155621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.156366 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.156371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.157206 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.157215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.157886 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.157890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.157999 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.158023 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 09:44:15.158137 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 09:44:15.158222 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":302,"thread_start_us":181,"threads_started":1}
I20260812 09:44:15.159101 23470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9321e65aaf594043b027dafea25c5f04" candidate_uuid: "af3fc285a9c04ce7b7b788196de36c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd89a1e671274fb4ac30bfea61801370" is_pre_election: true
I20260812 09:44:15.159245 23633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9321e65aaf594043b027dafea25c5f04" candidate_uuid: "af3fc285a9c04ce7b7b788196de36c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157c462b5f3a4165b8e2fb67c519a184" is_pre_election: true
I20260812 09:44:15.159443 23470 raft_consensus.cc:2454] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3fc285a9c04ce7b7b788196de36c46 in term 0.
I20260812 09:44:15.159576 23633 raft_consensus.cc:2454] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3fc285a9c04ce7b7b788196de36c46 in term 0.
I20260812 09:44:15.160017 23715 leader_election.cc:302] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 157c462b5f3a4165b8e2fb67c519a184, af3fc285a9c04ce7b7b788196de36c46, bd89a1e671274fb4ac30bfea61801370; no voters:
I20260812 09:44:15.160331 23865 raft_consensus.cc:2781] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:44:15.160424 23865 raft_consensus.cc:482] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:44:15.160482 23865 raft_consensus.cc:3037] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:15.161513 23865 raft_consensus.cc:504] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.161948 23865 leader_election.cc:288] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [CANDIDATE]: Term 1 election: Requested vote from peers 157c462b5f3a4165b8e2fb67c519a184 (127.3.147.2:46757), bd89a1e671274fb4ac30bfea61801370 (127.3.147.1:43613)
I20260812 09:44:15.162240 23633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9321e65aaf594043b027dafea25c5f04" candidate_uuid: "af3fc285a9c04ce7b7b788196de36c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "157c462b5f3a4165b8e2fb67c519a184"
I20260812 09:44:15.162360 23633 raft_consensus.cc:3037] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:15.162395 23470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9321e65aaf594043b027dafea25c5f04" candidate_uuid: "af3fc285a9c04ce7b7b788196de36c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd89a1e671274fb4ac30bfea61801370"
I20260812 09:44:15.162532 23470 raft_consensus.cc:3037] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:15.163445 23633 raft_consensus.cc:2454] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3fc285a9c04ce7b7b788196de36c46 in term 1.
I20260812 09:44:15.163648 23470 raft_consensus.cc:2454] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3fc285a9c04ce7b7b788196de36c46 in term 1.
I20260812 09:44:15.163767 23711 leader_election.cc:302] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 157c462b5f3a4165b8e2fb67c519a184, af3fc285a9c04ce7b7b788196de36c46; no voters:
I20260812 09:44:15.163944 23865 raft_consensus.cc:2781] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:15.164197 23865 raft_consensus.cc:686] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 1 LEADER]: Becoming Leader. State: Replica: af3fc285a9c04ce7b7b788196de36c46, State: Running, Role: LEADER
I20260812 09:44:15.164551 23865 consensus_queue.cc:237] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } }
I20260812 09:44:15.167539 23303 catalog_manager.cc:5495] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 reported cstate change: term changed from 0 to 1, leader changed from <none> to af3fc285a9c04ce7b7b788196de36c46 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "af3fc285a9c04ce7b7b788196de36c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af3fc285a9c04ce7b7b788196de36c46" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 42513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 } health_report { overall_health: UNKNOWN } } }
I20260812 09:44:15.200331 23849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.196588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33066 (local address 127.3.147.3:42513)
0812 09:44:15.196740 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:15.196745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:15.196760 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:44:15.196931 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:15.196933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:15.196992 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:15.197102 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:15.197392 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.197894 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.198774 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.198987 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.199090 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.199447 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:15.199862 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:15.199993 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.200063 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.200121 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 09:44:15.202917 23837 tablet.cc:2369] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:15.206360 23633 raft_consensus.cc:1261] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 1 FOLLOWER]: Refusing update from remote peer af3fc285a9c04ce7b7b788196de36c46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:15.206391 23470 raft_consensus.cc:1261] T 9321e65aaf594043b027dafea25c5f04 P bd89a1e671274fb4ac30bfea61801370 [term 1 FOLLOWER]: Refusing update from remote peer af3fc285a9c04ce7b7b788196de36c46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:15.207082 23865 consensus_queue.cc:1035] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd89a1e671274fb4ac30bfea61801370" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 43613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:15.207283 23869 consensus_queue.cc:1035] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "157c462b5f3a4165b8e2fb67c519a184" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 09:44:15.217281 23689 tablet.cc:2369] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:15.219383 23873 mvcc.cc:205] Tried to move back new op lower bound from 7317618094918053888 to 7317618094756724736. Current Snapshot: MvccSnapshot[applied={T|T < 7317618094918053888}]
I20260812 09:44:15.221261 23874 mvcc.cc:205] Tried to move back new op lower bound from 7317618094918053888 to 7317618094756724736. Current Snapshot: MvccSnapshot[applied={T|T < 7317618094918053888}]
I20260812 09:44:15.221274 23872 mvcc.cc:205] Tried to move back new op lower bound from 7317618094918053888 to 7317618094756724736. Current Snapshot: MvccSnapshot[applied={T|T < 7317618094918053888 or (T in {7317618094918053888})}]
I20260812 09:44:15.227411 23305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:15.227584 23305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:15.230090 23305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0043979aff174a97b53d40fedd492f49: Sending DeleteTablet for 3 replicas of tablet 9321e65aaf594043b027dafea25c5f04
I20260812 09:44:15.230964 23611 tablet_service.cc:1514] Processing DeleteTablet for tablet 9321e65aaf594043b027dafea25c5f04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:15 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:57322
I20260812 09:44:15.230962 23762 tablet_service.cc:1514] Processing DeleteTablet for tablet 9321e65aaf594043b027dafea25c5f04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:15 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:33050
I20260812 09:44:15.231468 23882 tablet_replica.cc:330] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: stopping tablet replica
I20260812 09:44:15.231451 23448 tablet_service.cc:1514] Processing DeleteTablet for tablet 9321e65aaf594043b027dafea25c5f04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:15 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:60824
I20260812 09:44:15.231798 23882 raft_consensus.cc:2229] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:15.232132 23882 raft_consensus.cc:2258] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:15.232972 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 23376
I20260812 09:44:15.236369 23883 tablet_replica.cc:330] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: stopping tablet replica
I20260812 09:44:15.236718 23883 raft_consensus.cc:2229] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:44:15.237071 23882 ts_tablet_manager.cc:1905] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:15.237128 23883 raft_consensus.cc:2258] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:15.238643 23883 ts_tablet_manager.cc:1905] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:15.240726 23882 ts_tablet_manager.cc:1918] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:15.240833 23882 log.cc:1196] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/9321e65aaf594043b027dafea25c5f04
I20260812 09:44:15.241196 23882 ts_tablet_manager.cc:1939] T 9321e65aaf594043b027dafea25c5f04 P 157c462b5f3a4165b8e2fb67c519a184: Deleting consensus metadata
I20260812 09:44:15.241539 23883 ts_tablet_manager.cc:1918] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:15.241686 23883 log.cc:1196] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/9321e65aaf594043b027dafea25c5f04
I20260812 09:44:15.242266 23883 ts_tablet_manager.cc:1939] T 9321e65aaf594043b027dafea25c5f04 P af3fc285a9c04ce7b7b788196de36c46: Deleting consensus metadata
I20260812 09:44:15.242570 23287 catalog_manager.cc:4841] TS 157c462b5f3a4165b8e2fb67c519a184 (127.3.147.2:46757): tablet 9321e65aaf594043b027dafea25c5f04 (table test-table [id=935a3c158a2b40b59f5a8e84e66f4e31]) successfully deleted
I20260812 09:44:15.243374 23287 catalog_manager.cc:4841] TS af3fc285a9c04ce7b7b788196de36c46 (127.3.147.3:42513): tablet 9321e65aaf594043b027dafea25c5f04 (table test-table [id=935a3c158a2b40b59f5a8e84e66f4e31]) successfully deleted
W20260812 09:44:15.244138 23290 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:43613 (error 108)
I20260812 09:44:15.245059 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 23534
I20260812 09:44:15.245235 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.244926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:43613 (local address 127.0.0.1:60830)
0812 09:44:15.245093 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:15.245159 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:43613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260812 09:44:15.246848 23290 catalog_manager.cc:4584] TS bd89a1e671274fb4ac30bfea61801370 (127.3.147.1:43613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9321e65aaf594043b027dafea25c5f04: Network error: Client connection negotiation failed: client connection to 127.3.147.1:43613: connect: Connection refused (error 111)
I20260812 09:44:15.254904 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 23693
I20260812 09:44:15.264074 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 23267
2026-08-12T09:44:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2592 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T09:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:15Z Disabled control of system clock
I20260812 09:44:15.298054 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43201
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:39361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:43201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 09:44:15.408327 23889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:15.408576 23889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:15.408612 23889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:15.412539 23889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:15.412587 23889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:15.412602 23889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:15.412618 23889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:15.412632 23889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:15.417279 23889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:43201
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:15.418228 23889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:15.419549 23889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:15.428035 23897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:15.428104 23895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:15.428121 23899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:15.428054 23889 server_base.cc:1034] running on GCE node
I20260812 09:44:15.428905 23889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:15.429961 23889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:15.431156 23889 hybrid_clock.cc:648] HybridClock initialized: now 1786527855431134 us; error 32 us; skew 500 ppm
I20260812 09:44:15.433056 23889 webserver.cc:460] Webserver started at http://127.3.147.62:43131/ using document root <none> and password file <none>
I20260812 09:44:15.433650 23889 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:15.433743 23889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:15.433972 23889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:15.435981 23889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "10c850f818094fd8a9a12140ccf8f4ce"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "a678d9d645816c8d964a18853a7036bb"
server_key_iv: "fa32e2e08e2f252c11114dfa994792ce"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.436594 23889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "10c850f818094fd8a9a12140ccf8f4ce"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "a678d9d645816c8d964a18853a7036bb"
server_key_iv: "fa32e2e08e2f252c11114dfa994792ce"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.440316 23889 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:44:15.442740 23905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:15.444017 23889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:15.444209 23889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "10c850f818094fd8a9a12140ccf8f4ce"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "a678d9d645816c8d964a18853a7036bb"
server_key_iv: "fa32e2e08e2f252c11114dfa994792ce"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.444339 23889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:15.472730 23889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:15.473564 23889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:15.473773 23889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:15.481590 23889 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:43201
I20260812 09:44:15.481608 23967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:43201 every 8 connection(s)
I20260812 09:44:15.482609 23889 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:15.483760 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 23889
I20260812 09:44:15.483896 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:15.484299 3660 external_mini_cluster.cc:1428] Setting key 8c52f3fc6fab46a7bc6032af105a1c91
I20260812 09:44:15.486086 23969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:15.492187 23969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce: Bootstrap starting.
I20260812 09:44:15.493793 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.485735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.3.147.62:43201)
0812 09:44:15.486410 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:15.486428 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:15.486466 (+ 38us) server_negotiation.cc:407] Connection header received
0812 09:44:15.487126 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:15.487142 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:15.487512 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:15.487794 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:15.488128 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.489221 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.490415 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.490880 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.491004 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.491881 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:15.491886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:15.491897 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:15.492438 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:15.492497 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:15.492538 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:15.492629 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.492793 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.493105 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":165,"threads_started":1}
I20260812 09:44:15.494526 23969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:15.495301 23969 log.cc:824] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:15.497131 23969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce: No bootstrap required, opened a new log
I20260812 09:44:15.500435 23969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c850f818094fd8a9a12140ccf8f4ce" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43201 } }
I20260812 09:44:15.500661 23969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:15.500747 23969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c850f818094fd8a9a12140ccf8f4ce, State: Initialized, Role: FOLLOWER
I20260812 09:44:15.501178 23969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c850f818094fd8a9a12140ccf8f4ce" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43201 } }
I20260812 09:44:15.501327 23969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:15.501413 23969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:15.501508 23969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:15.502344 23969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c850f818094fd8a9a12140ccf8f4ce" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43201 } }
I20260812 09:44:15.502662 23969 leader_election.cc:302] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c850f818094fd8a9a12140ccf8f4ce; no voters:
I20260812 09:44:15.502957 23969 leader_election.cc:288] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:15.503084 23976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:15.503377 23976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [term 1 LEADER]: Becoming Leader. State: Replica: 10c850f818094fd8a9a12140ccf8f4ce, State: Running, Role: LEADER
I20260812 09:44:15.503685 23976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c850f818094fd8a9a12140ccf8f4ce" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43201 } }
I20260812 09:44:15.504189 23969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:15.505437 23977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10c850f818094fd8a9a12140ccf8f4ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c850f818094fd8a9a12140ccf8f4ce" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43201 } } }
I20260812 09:44:15.505569 23977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:15.505931 23978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c850f818094fd8a9a12140ccf8f4ce. Latest consensus state: current_term: 1 leader_uuid: "10c850f818094fd8a9a12140ccf8f4ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c850f818094fd8a9a12140ccf8f4ce" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43201 } } }
I20260812 09:44:15.506032 23978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:15.506150 23986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:15.509320 23986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:15.514775 23986 catalog_manager.cc:1347] Generated new cluster ID: 7723a7be895c45b3b0b0d9270d925032
I20260812 09:44:15.514851 23986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:15.533237 23986 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:15.534135 23986 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:15.540684 23986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce: Generated new TSK 0
I20260812 09:44:15.541350 23986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:15.544073 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43201
--builtin_ntp_servers=127.3.147.20:39361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 09:44:15.666221 23997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:15.666462 23997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:15.666499 23997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:15.670441 23997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:15.670486 23997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:15.670538 23997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:44:15.675127 23997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:15.676160 23997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:15.677608 23997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:15.686997 24006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:15.686956 24003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:15.686960 24004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:15.687259 23997 server_base.cc:1034] running on GCE node
I20260812 09:44:15.687726 23997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:15.688432 23997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:15.689620 23997 hybrid_clock.cc:648] HybridClock initialized: now 1786527855689600 us; error 29 us; skew 500 ppm
I20260812 09:44:15.691357 23997 webserver.cc:460] Webserver started at http://127.3.147.1:37723/ using document root <none> and password file <none>
I20260812 09:44:15.691906 23997 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:15.691998 23997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:15.692309 23997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:15.694380 23997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "a803cc5c55c04b7b8cb8e3989f44932c"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "8bcdc96102ef4b0d7296ed4a5b955848"
server_key_iv: "e6ef21dfb29e56bd893f2b114d503ff8"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.694940 23997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "a803cc5c55c04b7b8cb8e3989f44932c"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "8bcdc96102ef4b0d7296ed4a5b955848"
server_key_iv: "e6ef21dfb29e56bd893f2b114d503ff8"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.698452 23997 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260812 09:44:15.700732 24012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:15.701885 23997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:15.702019 23997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "a803cc5c55c04b7b8cb8e3989f44932c"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "8bcdc96102ef4b0d7296ed4a5b955848"
server_key_iv: "e6ef21dfb29e56bd893f2b114d503ff8"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.702126 23997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:15.735896 23997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:15.736768 23997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:15.736982 23997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:15.737628 23997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:15.738610 23997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:15.738690 23997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:15.738761 23997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:15.738797 23997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:15.748247 23997 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:40591
I20260812 09:44:15.748306 24141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:40591 every 8 connection(s)
I20260812 09:44:15.749217 23997 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:44:15.752314 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 23997
I20260812 09:44:15.752450 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:44:15.752797 3660 external_mini_cluster.cc:1428] Setting key a1e7e34b28c5612758bcc76071bf7262
I20260812 09:44:15.755059 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43201
--builtin_ntp_servers=127.3.147.20:39361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 09:44:15.759830 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.750987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:38857 (local address 127.3.147.62:43201)
0812 09:44:15.751127 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:15.751131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:15.752248 (+ 1117us) server_negotiation.cc:407] Connection header received
0812 09:44:15.754046 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:15.754049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:15.754145 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:15.754246 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:15.755459 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.755990 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.757499 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.757918 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.758055 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.759161 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:15.759168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:15.759174 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:15.759193 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:15.759220 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:15.759233 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:15.759318 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.759470 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.759565 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:44:15.760659 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.751347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43201 (local address 127.3.147.1:38857)
0812 09:44:15.752019 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:15.752078 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:15.753838 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:15.754474 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:15.754483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:15.754585 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:15.755299 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.755323 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.756129 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.756159 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.757333 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.757342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.758096 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.758098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.758238 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.758896 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:15.758898 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:15.758938 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:15.758999 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:15.759324 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:15.759331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:15.759336 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 09:44:15.759428 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 09:44:15.759647 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":122,"threads_started":1}
I20260812 09:44:15.762063 24142 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43201
I20260812 09:44:15.762342 24142 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:15.763044 24142 heartbeater.cc:511] Master 127.3.147.62:43201 requested a full tablet report, sending...
I20260812 09:44:15.764906 23925 ts_manager.cc:194] Registered new tserver with Master: a803cc5c55c04b7b8cb8e3989f44932c (127.3.147.1:40591)
I20260812 09:44:15.766673 23925 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.1:38857
W20260812 09:44:15.870230 24146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:15.870462 24146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:15.870497 24146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:15.874286 24146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:15.874328 24146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:15.874378 24146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:44:15.878808 24146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:15.879730 24146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:15.881048 24146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:15.890836 24152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:15.890826 24156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:15.890933 24153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:15.891163 24146 server_base.cc:1034] running on GCE node
I20260812 09:44:15.891649 24146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:15.892385 24146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:15.893577 24146 hybrid_clock.cc:648] HybridClock initialized: now 1786527855893563 us; error 34 us; skew 500 ppm
I20260812 09:44:15.895452 24146 webserver.cc:460] Webserver started at http://127.3.147.2:35371/ using document root <none> and password file <none>
I20260812 09:44:15.896044 24146 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:15.896142 24146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:15.896399 24146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:15.898491 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "e44daea00c1b470a8075eaea4bb5abb1"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "4f9361e8e62180da4f4201bce5f23d61"
server_key_iv: "e17c178025b57604e2a832011f3b8178"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.899060 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "e44daea00c1b470a8075eaea4bb5abb1"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "4f9361e8e62180da4f4201bce5f23d61"
server_key_iv: "e17c178025b57604e2a832011f3b8178"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.902796 24146 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 09:44:15.905445 24162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:15.906805 24146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:44:15.906951 24146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "e44daea00c1b470a8075eaea4bb5abb1"
format_stamp: "Formatted at 2026-08-12 09:44:15 on dist-test-slave-cbbz"
server_key: "4f9361e8e62180da4f4201bce5f23d61"
server_key_iv: "e17c178025b57604e2a832011f3b8178"
server_key_version: "encryptionkey@0"
I20260812 09:44:15.907060 24146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:15.929179 24146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:15.929822 24146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:15.930016 24146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:15.930588 24146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:15.931514 24146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:15.931584 24146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:15.931661 24146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:15.931694 24146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:15.940917 24146 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:34273
I20260812 09:44:15.940940 24297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:34273 every 8 connection(s)
I20260812 09:44:15.941849 24146 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:44:15.942692 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 24146
I20260812 09:44:15.942840 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:44:15.943153 3660 external_mini_cluster.cc:1428] Setting key 65b94bc2cc0baaf065682b96cfd8174b
I20260812 09:44:15.945283 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43201
--builtin_ntp_servers=127.3.147.20:39361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 09:44:15.952514 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.943812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:58145 (local address 127.3.147.62:43201)
0812 09:44:15.943963 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:15.943967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:15.945930 (+ 1963us) server_negotiation.cc:407] Connection header received
0812 09:44:15.947247 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:15.947252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:15.947313 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:15.947411 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:15.948425 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.949020 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.950453 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.950687 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.950796 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.951944 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:15.951948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:15.951951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:15.951963 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:15.951984 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:15.952000 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:15.952092 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:15.952298 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 09:44:15.952365 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 09:44:15.953325 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:15.944255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43201 (local address 127.3.147.2:58145)
0812 09:44:15.945787 (+ 1532us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:15.945827 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:15.947043 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:15.947563 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:15.947570 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:15.947652 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:15.948270 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.948285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.949141 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.949145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.950325 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:15.950336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:15.950861 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:15.950868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:15.951035 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:15.951675 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:15.951677 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:15.951716 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:15.951783 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:15.952107 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:15.952117 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:15.952122 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 09:44:15.952279 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 09:44:15.952545 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":116,"threads_started":1}
I20260812 09:44:15.954715 24298 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43201
I20260812 09:44:15.954952 24298 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:15.955497 24298 heartbeater.cc:511] Master 127.3.147.62:43201 requested a full tablet report, sending...
I20260812 09:44:15.956672 23925 ts_manager.cc:194] Registered new tserver with Master: e44daea00c1b470a8075eaea4bb5abb1 (127.3.147.2:34273)
I20260812 09:44:15.957541 23925 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.2:58145
W20260812 09:44:16.060111 24301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:16.060427 24301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:16.060465 24301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:16.064402 24301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:16.064447 24301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:16.064498 24301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:44:16.069097 24301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:39361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:16.070010 24301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:16.071303 24301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:16.080770 24307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:16.080782 24310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:16.080782 24308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:16.081131 24301 server_base.cc:1034] running on GCE node
I20260812 09:44:16.081580 24301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:16.082149 24301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:16.083312 24301 hybrid_clock.cc:648] HybridClock initialized: now 1786527856083285 us; error 41 us; skew 500 ppm
I20260812 09:44:16.085210 24301 webserver.cc:460] Webserver started at http://127.3.147.3:37151/ using document root <none> and password file <none>
I20260812 09:44:16.085799 24301 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:16.085865 24301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:16.086115 24301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:16.088548 24301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b"
format_stamp: "Formatted at 2026-08-12 09:44:16 on dist-test-slave-cbbz"
server_key: "3b2a6dd07dde6905ed7eaac202ab4319"
server_key_iv: "bf2b3f0adb99c22502d8d411821b8378"
server_key_version: "encryptionkey@0"
I20260812 09:44:16.089186 24301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b"
format_stamp: "Formatted at 2026-08-12 09:44:16 on dist-test-slave-cbbz"
server_key: "3b2a6dd07dde6905ed7eaac202ab4319"
server_key_iv: "bf2b3f0adb99c22502d8d411821b8378"
server_key_version: "encryptionkey@0"
I20260812 09:44:16.093123 24301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 09:44:16.095633 24317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:16.096923 24301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:44:16.097021 24301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b"
format_stamp: "Formatted at 2026-08-12 09:44:16 on dist-test-slave-cbbz"
server_key: "3b2a6dd07dde6905ed7eaac202ab4319"
server_key_iv: "bf2b3f0adb99c22502d8d411821b8378"
server_key_version: "encryptionkey@0"
I20260812 09:44:16.097149 24301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:16.107687 24301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:16.108376 24301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:16.108563 24301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:16.109144 24301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:16.110141 24301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:16.110188 24301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:16.110251 24301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:16.110293 24301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:16.120354 24301 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:38643
I20260812 09:44:16.120445 24448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:38643 every 8 connection(s)
I20260812 09:44:16.121337 24301 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:44:16.122318 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 24301
I20260812 09:44:16.122431 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:44:16.122762 3660 external_mini_cluster.cc:1428] Setting key 110047fa57f4432fc75480e828816933
I20260812 09:44:16.131263 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.123552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:53933 (local address 127.3.147.62:43201)
0812 09:44:16.123707 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:16.123712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:16.125061 (+ 1349us) server_negotiation.cc:407] Connection header received
0812 09:44:16.126508 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:16.126511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:16.126567 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:16.126642 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:16.127560 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.128099 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.129314 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.129558 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.129690 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.130742 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:16.130745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:16.130748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:16.130761 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:16.130784 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:16.130793 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:16.130876 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:16.131056 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 09:44:16.131119 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 09:44:16.132126 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.124041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:43201 (local address 127.3.147.3:53933)
0812 09:44:16.124899 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:16.124954 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:16.126295 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:16.126826 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:16.126833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:16.126918 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:16.127421 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.127433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.128284 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.128287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.129177 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.129187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.129730 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.129734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.129922 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.130494 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:16.130496 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:16.130533 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:16.130594 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:16.130878 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:16.130891 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:16.130897 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 09:44:16.131043 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 09:44:16.131319 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":156,"threads_started":1}
I20260812 09:44:16.133337 24449 heartbeater.cc:347] Connected to a master server at 127.3.147.62:43201
I20260812 09:44:16.133546 24449 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:16.134017 24449 heartbeater.cc:511] Master 127.3.147.62:43201 requested a full tablet report, sending...
I20260812 09:44:16.135001 23925 ts_manager.cc:194] Registered new tserver with Master: d05588d0fb6d46dcb0852f14f5d7ba6b (127.3.147.3:38643)
I20260812 09:44:16.135592 23925 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.3:53933
I20260812 09:44:16.136897 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:44:16.142591 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.138915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42332 (local address 127.3.147.62:43201)
0812 09:44:16.139059 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:16.139063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:16.139260 (+ 197us) server_negotiation.cc:407] Connection header received
0812 09:44:16.139341 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:16.139343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:16.139380 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:16.139442 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:16.139624 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.140095 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.141101 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.141303 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.141374 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.142264 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:16.142266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:16.142269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:16.142278 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:16.142294 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:16.142301 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:16.142352 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:16.142408 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 09:44:16.142449 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 09:44:16.145519 23925 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:44:16.148558 23925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:44:16.163499 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.157864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55904 (local address 127.3.147.2:34273)
0812 09:44:16.158416 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:16.158428 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:16.158453 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:44:16.158530 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:16.158539 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:16.158744 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:16.159087 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:16.159265 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.160636 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.162176 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.162761 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.162896 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.163171 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:16.163232 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 09:44:16.163327 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":149,"threads_started":1}
I20260812 09:44:16.163494 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.157700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:34273 (local address 127.0.0.1:55904)
0812 09:44:16.158027 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:16.158041 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:16.158143 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:16.158898 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:16.158901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:16.158913 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:16.159145 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.159155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.160828 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.160831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.162042 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.162052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.162897 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.162901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.163015 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.163082 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 09:44:16.163199 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 09:44:16.163322 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":85,"thread_start_us":106,"threads_started":1}
I20260812 09:44:16.163823 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.157421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:38643 (local address 127.0.0.1:34974)
0812 09:44:16.158217 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:16.158232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:16.158343 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:16.159332 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:16.159334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:16.159348 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:16.159407 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.159410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.160483 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.160488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.162039 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.162050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.163312 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.163316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.163459 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.163531 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 09:44:16.163595 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 09:44:16.163672 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":149,"spinlock_wait_cycles":6400,"thread_start_us":102,"threads_started":1}
I20260812 09:44:16.163936 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.157092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44668 (local address 127.3.147.1:40591)
0812 09:44:16.157494 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:16.157507 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:16.157917 (+ 410us) server_negotiation.cc:407] Connection header received
0812 09:44:16.158247 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:16.158253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:16.158477 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:16.158628 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:16.159247 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.160611 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.162190 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.162981 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.163069 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.163242 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:16.163661 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 09:44:16.163755 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":170,"threads_started":1}
I20260812 09:44:16.164300 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.157888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.3.147.3:38643)
0812 09:44:16.158712 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:16.158725 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:16.158749 (+ 24us) server_negotiation.cc:407] Connection header received
0812 09:44:16.158840 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:16.158846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:16.159014 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:16.159178 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:16.159519 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.160369 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.162190 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.162981 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.163082 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.163242 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:16.163943 (+ 701us) server_negotiation.cc:299] Negotiation successful
0812 09:44:16.164075 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":138,"threads_started":1}
I20260812 09:44:16.165392 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.156943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:40591 (local address 127.0.0.1:44668)
0812 09:44:16.157765 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:16.157822 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:16.158035 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:16.158860 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:16.158869 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:16.158897 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:16.159101 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.159109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.160736 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.160739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.162039 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.162050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.163344 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.163346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.163461 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.163523 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 09:44:16.165164 (+ 1641us) client_negotiation.cc:240] Negotiation successful
0812 09:44:16.165249 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":132,"spinlock_wait_cycles":2432,"thread_start_us":81,"threads_started":1}
I20260812 09:44:16.166966 24214 tablet_service.cc:1467] Processing CreateTablet for tablet d4fe857b54cf494bae1386cbc48f35d8 (DEFAULT_TABLE table=test-table [id=60c5140c45c44521b44776ff62a016ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:16.167083 24371 tablet_service.cc:1467] Processing CreateTablet for tablet d4fe857b54cf494bae1386cbc48f35d8 (DEFAULT_TABLE table=test-table [id=60c5140c45c44521b44776ff62a016ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:16.167991 24214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fe857b54cf494bae1386cbc48f35d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:16.168267 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fe857b54cf494bae1386cbc48f35d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:16.168069 24067 tablet_service.cc:1467] Processing CreateTablet for tablet d4fe857b54cf494bae1386cbc48f35d8 (DEFAULT_TABLE table=test-table [id=60c5140c45c44521b44776ff62a016ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:16.169059 24067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fe857b54cf494bae1386cbc48f35d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:16.174883 24468 tablet_bootstrap.cc:492] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Bootstrap starting.
I20260812 09:44:16.176011 24469 tablet_bootstrap.cc:492] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: Bootstrap starting.
I20260812 09:44:16.177079 24468 tablet_bootstrap.cc:654] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:16.177078 24470 tablet_bootstrap.cc:492] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c: Bootstrap starting.
I20260812 09:44:16.177945 24468 log.cc:824] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:16.178591 24469 tablet_bootstrap.cc:654] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:16.179147 24470 tablet_bootstrap.cc:654] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:16.179395 24469 log.cc:824] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:16.179976 24470 log.cc:824] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:16.180172 24468 tablet_bootstrap.cc:492] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: No bootstrap required, opened a new log
I20260812 09:44:16.180398 24468 ts_tablet_manager.cc:1397] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:44:16.181603 24469 tablet_bootstrap.cc:492] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: No bootstrap required, opened a new log
I20260812 09:44:16.181775 24470 tablet_bootstrap.cc:492] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c: No bootstrap required, opened a new log
I20260812 09:44:16.181846 24469 ts_tablet_manager.cc:1397] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:44:16.182013 24470 ts_tablet_manager.cc:1397] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:44:16.184849 24468 raft_consensus.cc:348] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.185114 24468 raft_consensus.cc:374] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:16.185176 24468 raft_consensus.cc:729] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44daea00c1b470a8075eaea4bb5abb1, State: Initialized, Role: FOLLOWER
I20260812 09:44:16.185458 24470 raft_consensus.cc:348] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.185652 24468 consensus_queue.cc:260] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.185726 24470 raft_consensus.cc:374] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:16.185789 24470 raft_consensus.cc:729] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a803cc5c55c04b7b8cb8e3989f44932c, State: Initialized, Role: FOLLOWER
I20260812 09:44:16.186064 24469 raft_consensus.cc:348] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.186297 24469 raft_consensus.cc:374] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:16.186354 24469 raft_consensus.cc:729] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d05588d0fb6d46dcb0852f14f5d7ba6b, State: Initialized, Role: FOLLOWER
I20260812 09:44:16.186260 24470 consensus_queue.cc:260] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.186647 24468 ts_tablet_manager.cc:1428] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260812 09:44:16.186731 24298 heartbeater.cc:503] Master 127.3.147.62:43201 was elected leader, sending a full tablet report...
I20260812 09:44:16.187211 24142 heartbeater.cc:503] Master 127.3.147.62:43201 was elected leader, sending a full tablet report...
I20260812 09:44:16.187268 24470 ts_tablet_manager.cc:1428] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 09:44:16.187316 24469 consensus_queue.cc:260] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.188004 24449 heartbeater.cc:503] Master 127.3.147.62:43201 was elected leader, sending a full tablet report...
I20260812 09:44:16.188241 24469 ts_tablet_manager.cc:1428] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260812 09:44:16.192750 24299 tablet.cc:2369] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 09:44:16.250495 24143 tablet.cc:2369] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:16.287127 24474 raft_consensus.cc:482] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:16.287312 24474 raft_consensus.cc:504] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.288501 24474 leader_election.cc:288] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a803cc5c55c04b7b8cb8e3989f44932c (127.3.147.1:40591), d05588d0fb6d46dcb0852f14f5d7ba6b (127.3.147.3:38643)
I20260812 09:44:16.292404 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.288968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:46953 (local address 127.3.147.3:38643)
0812 09:44:16.289098 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:16.289103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:16.289119 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:44:16.289182 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:16.289186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:16.289236 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:16.289331 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:16.289649 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.290240 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.291331 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.291755 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.291856 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.291945 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:16.292228 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 09:44:16.292305 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:16.292395 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.288737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:38643 (local address 127.3.147.2:46953)
0812 09:44:16.288924 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:16.288942 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:16.289053 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:16.289365 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:16.289370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:16.289389 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:16.289532 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.289542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.290354 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.290357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.291204 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.291213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.291904 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.291908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.292003 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.292049 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 09:44:16.292137 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 09:44:16.292251 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":2}
I20260812 09:44:16.292515 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.288874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:46577 (local address 127.3.147.1:40591)
0812 09:44:16.289012 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:16.289018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:16.289341 (+ 323us) server_negotiation.cc:407] Connection header received
0812 09:44:16.289437 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:16.289440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:16.289494 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:16.289582 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:16.289861 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.290397 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.291412 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.291793 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.291911 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.291969 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:16.292320 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 09:44:16.292388 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:16.292558 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.288737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:40591 (local address 127.3.147.2:46577)
0812 09:44:16.289227 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:16.289241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:16.289354 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:16.289616 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:16.289619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:16.289638 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:16.289749 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.289755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.290503 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.290505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.291289 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:16.291300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.291945 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.291950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:16.292103 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.292130 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 09:44:16.292305 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 09:44:16.292374 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":154,"thread_start_us":140,"threads_started":1}
I20260812 09:44:16.293210 24394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fe857b54cf494bae1386cbc48f35d8" candidate_uuid: "e44daea00c1b470a8075eaea4bb5abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" is_pre_election: true
I20260812 09:44:16.293205 24090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fe857b54cf494bae1386cbc48f35d8" candidate_uuid: "e44daea00c1b470a8075eaea4bb5abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" is_pre_election: true
I20260812 09:44:16.293500 24394 raft_consensus.cc:2454] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44daea00c1b470a8075eaea4bb5abb1 in term 0.
I20260812 09:44:16.293524 24090 raft_consensus.cc:2454] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44daea00c1b470a8075eaea4bb5abb1 in term 0.
I20260812 09:44:16.293993 24166 leader_election.cc:302] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d05588d0fb6d46dcb0852f14f5d7ba6b, e44daea00c1b470a8075eaea4bb5abb1; no voters:
I20260812 09:44:16.294291 24474 raft_consensus.cc:2781] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:44:16.294373 24474 raft_consensus.cc:482] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:44:16.294404 24474 raft_consensus.cc:3037] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:16.295442 24474 raft_consensus.cc:504] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.295816 24474 leader_election.cc:288] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [CANDIDATE]: Term 1 election: Requested vote from peers a803cc5c55c04b7b8cb8e3989f44932c (127.3.147.1:40591), d05588d0fb6d46dcb0852f14f5d7ba6b (127.3.147.3:38643)
I20260812 09:44:16.296161 24090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fe857b54cf494bae1386cbc48f35d8" candidate_uuid: "e44daea00c1b470a8075eaea4bb5abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a803cc5c55c04b7b8cb8e3989f44932c"
I20260812 09:44:16.296221 24394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fe857b54cf494bae1386cbc48f35d8" candidate_uuid: "e44daea00c1b470a8075eaea4bb5abb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b"
I20260812 09:44:16.296288 24090 raft_consensus.cc:3037] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:16.296355 24394 raft_consensus.cc:3037] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:16.297297 24090 raft_consensus.cc:2454] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44daea00c1b470a8075eaea4bb5abb1 in term 1.
I20260812 09:44:16.297505 24394 raft_consensus.cc:2454] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44daea00c1b470a8075eaea4bb5abb1 in term 1.
I20260812 09:44:16.297665 24163 leader_election.cc:302] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a803cc5c55c04b7b8cb8e3989f44932c, e44daea00c1b470a8075eaea4bb5abb1; no voters:
I20260812 09:44:16.297832 24474 raft_consensus.cc:2781] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:16.298007 24474 raft_consensus.cc:686] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 1 LEADER]: Becoming Leader. State: Replica: e44daea00c1b470a8075eaea4bb5abb1, State: Running, Role: LEADER
I20260812 09:44:16.298314 24474 consensus_queue.cc:237] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } }
I20260812 09:44:16.301322 23924 catalog_manager.cc:5495] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e44daea00c1b470a8075eaea4bb5abb1 (127.3.147.2). New cstate: current_term: 1 leader_uuid: "e44daea00c1b470a8075eaea4bb5abb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44daea00c1b470a8075eaea4bb5abb1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 } health_report { overall_health: UNKNOWN } } }
I20260812 09:44:16.312773 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.309369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.3.147.2:34273)
0812 09:44:16.309489 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:16.309496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:16.309511 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:44:16.309691 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:16.309694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:16.309741 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:16.309841 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:16.310069 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.310521 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.311380 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:16.311577 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:16.311674 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:16.311958 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:16.312411 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:16.312527 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:16.312587 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 09:44:16.312630 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 09:44:16.318174 24394 raft_consensus.cc:1261] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 1 FOLLOWER]: Refusing update from remote peer e44daea00c1b470a8075eaea4bb5abb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:16.318789 24090 raft_consensus.cc:1261] T d4fe857b54cf494bae1386cbc48f35d8 P a803cc5c55c04b7b8cb8e3989f44932c [term 1 FOLLOWER]: Refusing update from remote peer e44daea00c1b470a8075eaea4bb5abb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:16.318840 24474 consensus_queue.cc:1035] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d05588d0fb6d46dcb0852f14f5d7ba6b" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:16.319396 24474 consensus_queue.cc:1035] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a803cc5c55c04b7b8cb8e3989f44932c" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:16.326143 24481 mvcc.cc:205] Tried to move back new op lower bound from 7317618099472478208 to 7317618099400187904. Current Snapshot: MvccSnapshot[applied={T|T < 7317618099472478208}]
I20260812 09:44:16.327118 24483 mvcc.cc:205] Tried to move back new op lower bound from 7317618099472478208 to 7317618099400187904. Current Snapshot: MvccSnapshot[applied={T|T < 7317618099472478208}]
I20260812 09:44:16.327284 24482 mvcc.cc:205] Tried to move back new op lower bound from 7317618099472478208 to 7317618099400187904. Current Snapshot: MvccSnapshot[applied={T|T < 7317618099472478208}]
I20260812 09:44:16.334017 23925 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:16.334173 23925 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:16.336395 23925 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10c850f818094fd8a9a12140ccf8f4ce: Sending DeleteTablet for 3 replicas of tablet d4fe857b54cf494bae1386cbc48f35d8
I20260812 09:44:16.337225 24067 tablet_service.cc:1514] Processing DeleteTablet for tablet d4fe857b54cf494bae1386cbc48f35d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:16 UTC) from {username='slave'} at 127.0.0.1:44668
I20260812 09:44:16.337263 24371 tablet_service.cc:1514] Processing DeleteTablet for tablet d4fe857b54cf494bae1386cbc48f35d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:16 UTC) from {username='slave'} at 127.0.0.1:34974
I20260812 09:44:16.337702 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 23997
I20260812 09:44:16.337750 24214 tablet_service.cc:1514] Processing DeleteTablet for tablet d4fe857b54cf494bae1386cbc48f35d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:16 UTC) from {username='slave'} at 127.0.0.1:55904
I20260812 09:44:16.339597 24495 tablet_replica.cc:330] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: stopping tablet replica
I20260812 09:44:16.339908 24495 raft_consensus.cc:2229] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:44:16.340272 24495 raft_consensus.cc:2258] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:16.340590 24494 tablet_replica.cc:330] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: stopping tablet replica
I20260812 09:44:16.340844 24494 raft_consensus.cc:2229] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:16.341125 24494 raft_consensus.cc:2258] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:16.342682 24494 ts_tablet_manager.cc:1905] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:16.345854 24494 ts_tablet_manager.cc:1918] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:16.345956 24494 log.cc:1196] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/d4fe857b54cf494bae1386cbc48f35d8
I20260812 09:44:16.346289 24494 ts_tablet_manager.cc:1939] T d4fe857b54cf494bae1386cbc48f35d8 P d05588d0fb6d46dcb0852f14f5d7ba6b: Deleting consensus metadata
I20260812 09:44:16.347432 24495 ts_tablet_manager.cc:1905] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:16.347922 23910 catalog_manager.cc:4841] TS d05588d0fb6d46dcb0852f14f5d7ba6b (127.3.147.3:38643): tablet d4fe857b54cf494bae1386cbc48f35d8 (table test-table [id=60c5140c45c44521b44776ff62a016ef]) successfully deleted
I20260812 09:44:16.350142 24495 ts_tablet_manager.cc:1918] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:16.350253 24495 log.cc:1196] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/d4fe857b54cf494bae1386cbc48f35d8
I20260812 09:44:16.350560 24495 ts_tablet_manager.cc:1939] T d4fe857b54cf494bae1386cbc48f35d8 P e44daea00c1b470a8075eaea4bb5abb1: Deleting consensus metadata
W20260812 09:44:16.350795 23906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:40591 (error 108)
I20260812 09:44:16.351696 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 24146
I20260812 09:44:16.351682 23906 catalog_manager.cc:4841] TS e44daea00c1b470a8075eaea4bb5abb1 (127.3.147.2:34273): tablet d4fe857b54cf494bae1386cbc48f35d8 (table test-table [id=60c5140c45c44521b44776ff62a016ef]) successfully deleted
I20260812 09:44:16.351869 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:16.351484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:40591 (local address 127.0.0.1:44684)
0812 09:44:16.351680 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:16.351747 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:40591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 09:44:16.355557 23906 catalog_manager.cc:4584] TS a803cc5c55c04b7b8cb8e3989f44932c (127.3.147.1:40591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4fe857b54cf494bae1386cbc48f35d8: Network error: Client connection negotiation failed: client connection to 127.3.147.1:40591: connect: Connection refused (error 111)
I20260812 09:44:16.360867 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 24301
I20260812 09:44:16.367303 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 23889
2026-08-12T09:44:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1101 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:16 dist-test-slave-cbbz krb5kdc[24499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:16 dist-test-slave-cbbz krb5kdc[24499](info): set up 2 sockets
Aug 12 09:44:16 dist-test-slave-cbbz krb5kdc[24499](info): commencing operation
krb5kdc: starting...
W20260812 09:44:17.484017 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:17 dist-test-slave-cbbz krb5kdc[24499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:17.609005 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:45045
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:45045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:17.717912 24515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:17.718160 24515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:17.718197 24515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:17.722045 24515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:17.722100 24515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:17.722118 24515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:17.722136 24515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:17.722150 24515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:17.727351 24515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:45045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:45045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:17.728392 24515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:17.729375 24515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:17.738209 24521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:17.738194 24522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:17.738273 24515 server_base.cc:1034] running on GCE node
W20260812 09:44:17.738194 24524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:17.739009 24515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:17.740077 24515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:17.741284 24515 hybrid_clock.cc:648] HybridClock initialized: now 1786527857741266 us; error 31 us; skew 500 ppm
Aug 12 09:44:17 dist-test-slave-cbbz krb5kdc[24499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527857, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:17.748268 24515 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:17.749141 24515 webserver.cc:460] Webserver started at http://127.3.147.62:45029/ using document root <none> and password file <none>
I20260812 09:44:17.749709 24515 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:17.749805 24515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:17.750043 24515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:17.752067 24515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "c214623cb45c41679824f6d7da2dc162"
format_stamp: "Formatted at 2026-08-12 09:44:17 on dist-test-slave-cbbz"
server_key: "b2c4bf823e9af5702309e23f82a50b63"
server_key_iv: "7feffe1bc9127aa9dd0aba2d79a8b169"
server_key_version: "encryptionkey@0"
I20260812 09:44:17.752688 24515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "c214623cb45c41679824f6d7da2dc162"
format_stamp: "Formatted at 2026-08-12 09:44:17 on dist-test-slave-cbbz"
server_key: "b2c4bf823e9af5702309e23f82a50b63"
server_key_iv: "7feffe1bc9127aa9dd0aba2d79a8b169"
server_key_version: "encryptionkey@0"
I20260812 09:44:17.756302 24515 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:17.758529 24531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:17.759665 24515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:17.759830 24515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "c214623cb45c41679824f6d7da2dc162"
format_stamp: "Formatted at 2026-08-12 09:44:17 on dist-test-slave-cbbz"
server_key: "b2c4bf823e9af5702309e23f82a50b63"
server_key_iv: "7feffe1bc9127aa9dd0aba2d79a8b169"
server_key_version: "encryptionkey@0"
I20260812 09:44:17.759940 24515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:17.769948 24515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:17.772773 24515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:17.772976 24515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:17.780042 24515 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:45045
I20260812 09:44:17.780056 24585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:45045 every 8 connection(s)
I20260812 09:44:17.781080 24515 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:17.783830 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:17.784745 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 24515
I20260812 09:44:17.784909 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:17.785300 3660 external_mini_cluster.cc:1428] Setting key 98ee95a814b0df5a0923c815a88f2149
I20260812 09:44:17.790280 24588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162: Bootstrap starting.
I20260812 09:44:17.792637 24588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:17.793421 24588 log.cc:824] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:44:17 dist-test-slave-cbbz krb5kdc[24499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:17.795238 24588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162: No bootstrap required, opened a new log
I20260812 09:44:17.798573 24588 raft_consensus.cc:348] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c214623cb45c41679824f6d7da2dc162" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45045 } }
I20260812 09:44:17.798842 24588 raft_consensus.cc:374] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:17.798924 24588 raft_consensus.cc:729] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c214623cb45c41679824f6d7da2dc162, State: Initialized, Role: FOLLOWER
I20260812 09:44:17.799361 24588 consensus_queue.cc:260] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c214623cb45c41679824f6d7da2dc162" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45045 } }
I20260812 09:44:17.799471 24588 raft_consensus.cc:388] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:17.799548 24588 raft_consensus.cc:482] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:17.799651 24588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:17.799851 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:17.786753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.3.147.62:45045)
0812 09:44:17.787214 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:17.787229 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:17.787270 (+ 41us) server_negotiation.cc:407] Connection header received
0812 09:44:17.787915 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:17.787930 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:17.788310 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:17.788638 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:17.789478 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:17.790491 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:17.791310 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:17.791743 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:17.791907 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:17.794649 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:17.794655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:17.794667 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:17.794694 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:17.796613 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:17.797131 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:17.797138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:17.797142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:17.797258 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:17.797597 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:17.797601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:17.797605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:17.797983 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:17.798240 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:17.798533 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 09:44:17.798853 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":142,"threads_started":1}
I20260812 09:44:17.800647 24588 raft_consensus.cc:504] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c214623cb45c41679824f6d7da2dc162" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45045 } }
I20260812 09:44:17.800964 24588 leader_election.cc:302] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c214623cb45c41679824f6d7da2dc162; no voters:
I20260812 09:44:17.801304 24588 leader_election.cc:288] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:17.801416 24595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:17.801664 24595 raft_consensus.cc:686] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [term 1 LEADER]: Becoming Leader. State: Replica: c214623cb45c41679824f6d7da2dc162, State: Running, Role: LEADER
I20260812 09:44:17.801990 24595 consensus_queue.cc:237] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c214623cb45c41679824f6d7da2dc162" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45045 } }
I20260812 09:44:17.802615 24588 sys_catalog.cc:564] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:17.803110 24596 sys_catalog.cc:455] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c214623cb45c41679824f6d7da2dc162. Latest consensus state: current_term: 1 leader_uuid: "c214623cb45c41679824f6d7da2dc162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c214623cb45c41679824f6d7da2dc162" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45045 } } }
I20260812 09:44:17.803239 24596 sys_catalog.cc:458] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:17.803627 24597 sys_catalog.cc:455] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c214623cb45c41679824f6d7da2dc162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c214623cb45c41679824f6d7da2dc162" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45045 } } }
I20260812 09:44:17.803704 24599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:17.803741 24597 sys_catalog.cc:458] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:17.806639 24599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:17.812340 24599 catalog_manager.cc:1347] Generated new cluster ID: 1d263184e85645ad8733fa8cc019539e
I20260812 09:44:17.812409 24599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:17.828017 24599 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:17.829022 24599 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:17.837414 24599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162: Generated new TSK 0
I20260812 09:44:17.838111 24599 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:44:17.872687 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:45045
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:17.983915 24617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:17.984179 24617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:17.984288 24617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:17.988408 24617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:17.988457 24617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:17.988546 24617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:44:17.993752 24617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:17.994813 24617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:17.995894 24617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:18.005811 24625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:18.005877 24622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:18.005834 24623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:18.006057 24617 server_base.cc:1034] running on GCE node
I20260812 09:44:18.006597 24617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:18.007246 24617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:18.008445 24617 hybrid_clock.cc:648] HybridClock initialized: now 1786527858008413 us; error 45 us; skew 500 ppm
Aug 12 09:44:18 dist-test-slave-cbbz krb5kdc[24499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527858, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:18.011252 24617 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:44:18.012050 24617 webserver.cc:460] Webserver started at http://127.3.147.1:45339/ using document root <none> and password file <none>
I20260812 09:44:18.012673 24617 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:18.012768 24617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:18.012995 24617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:18.014955 24617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "b7225bfd10144650a4d64f2e3074b510"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "d3ea3f4bf990425e0723271199e0a88a"
server_key_iv: "08460a234bc2d7b05351eb9fe92653a8"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.015529 24617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "b7225bfd10144650a4d64f2e3074b510"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "d3ea3f4bf990425e0723271199e0a88a"
server_key_iv: "08460a234bc2d7b05351eb9fe92653a8"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.019235 24617 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 09:44:18.021692 24633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.022915 24617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:44:18.023044 24617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "b7225bfd10144650a4d64f2e3074b510"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "d3ea3f4bf990425e0723271199e0a88a"
server_key_iv: "08460a234bc2d7b05351eb9fe92653a8"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.023161 24617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:18.054231 24617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:18.057643 24617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:18.057839 24617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:18.058472 24617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:18.059299 24617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:18.059366 24617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.059434 24617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:18.059474 24617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.068671 24617 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:37679
I20260812 09:44:18.068718 24757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:37679 every 8 connection(s)
I20260812 09:44:18.069607 24617 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
Aug 12 09:44:18 dist-test-slave-cbbz krb5kdc[24499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527858, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:18.079777 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 24617
I20260812 09:44:18.079917 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:44:18.080314 3660 external_mini_cluster.cc:1428] Setting key f9c01561d3ba68742d090d3bb3ca82a0
I20260812 09:44:18.083927 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.071394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:33107 (local address 127.3.147.62:45045)
0812 09:44:18.071541 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.071545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.072437 (+ 892us) server_negotiation.cc:407] Connection header received
0812 09:44:18.073798 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.073801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.073861 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.073971 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:18.075146 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.075693 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.076903 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.077148 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.077260 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.079895 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:18.079898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:18.079900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:18.079925 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:18.081692 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:18.082324 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:18.082327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:18.082328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:18.082405 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:18.082796 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:18.082798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:18.082799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:18.082962 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:18.083037 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.083692 (+ 655us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.083806 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 09:44:18.084954 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.071782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:45045 (local address 127.3.147.1:33107)
0812 09:44:18.072291 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.072335 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:18.073565 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:18.074151 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:18.074158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:18.074509 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:18.074988 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.075003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.075829 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.075832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.076769 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.076779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.077305 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.077309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.077487 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.078099 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:18.078115 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:18.079721 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:18.081826 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:18.081831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:18.081842 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:18.082209 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:18.082501 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:18.082507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:18.082512 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:18.082701 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:18.083080 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:18.083088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:18.083428 (+ 340us) client_negotiation.cc:769] Sending connection context
0812 09:44:18.083640 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 09:44:18.084020 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":125,"threads_started":1}
I20260812 09:44:18.086184 24758 heartbeater.cc:347] Connected to a master server at 127.3.147.62:45045
I20260812 09:44:18.086445 24758 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:18.086984 24758 heartbeater.cc:511] Master 127.3.147.62:45045 requested a full tablet report, sending...
I20260812 09:44:18.088546 24550 ts_manager.cc:194] Registered new tserver with Master: b7225bfd10144650a4d64f2e3074b510 (127.3.147.1:37679)
I20260812 09:44:18.089711 24550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:33107
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:44:18.110842 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:45045
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:18.223321 24763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:18.223589 24763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:18.223660 24763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:18.227777 24763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:18.227854 24763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:18.227942 24763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:44:18.233070 24763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:18.234090 24763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:18.235072 24763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:18.244575 24769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:18.244597 24770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:18.244874 24772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:18.245641 24763 server_base.cc:1034] running on GCE node
I20260812 09:44:18.246120 24763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:18.246882 24763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:18.248088 24763 hybrid_clock.cc:648] HybridClock initialized: now 1786527858248072 us; error 24 us; skew 500 ppm
Aug 12 09:44:18 dist-test-slave-cbbz krb5kdc[24499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527858, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:18.250910 24763 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:44:18.251740 24763 webserver.cc:460] Webserver started at http://127.3.147.2:39451/ using document root <none> and password file <none>
I20260812 09:44:18.252367 24763 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:18.252458 24763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:18.252694 24763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:18.254701 24763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "59c12c06fd484e96b91e2cd1f1677f01"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "d3809af635562096cc4bb3632a368249"
server_key_iv: "87908b59618f5a20fc860b8b0513df32"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.255252 24763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "59c12c06fd484e96b91e2cd1f1677f01"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "d3809af635562096cc4bb3632a368249"
server_key_iv: "87908b59618f5a20fc860b8b0513df32"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.258807 24763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 09:44:18.261102 24780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.262298 24763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:18.262460 24763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "59c12c06fd484e96b91e2cd1f1677f01"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "d3809af635562096cc4bb3632a368249"
server_key_iv: "87908b59618f5a20fc860b8b0513df32"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.262578 24763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:18.275818 24763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:18.278498 24763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:18.278652 24763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:18.279182 24763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:18.279922 24763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:18.279968 24763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.280017 24763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:18.280038 24763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.289402 24763 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:34395
I20260812 09:44:18.289424 24907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:34395 every 8 connection(s)
I20260812 09:44:18.290277 24763 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:44:18.297501 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 24763
I20260812 09:44:18.297675 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:44:18.297991 3660 external_mini_cluster.cc:1428] Setting key f9aab0dc1f7c0abce6619949001ca863
Aug 12 09:44:18 dist-test-slave-cbbz krb5kdc[24499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527858, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:18.304656 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.292175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:59869 (local address 127.3.147.62:45045)
0812 09:44:18.292303 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.292306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.293145 (+ 839us) server_negotiation.cc:407] Connection header received
0812 09:44:18.294442 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.294445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.294519 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.294602 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:18.295765 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.296393 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.297515 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.297863 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.297987 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.300524 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:18.300527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:18.300530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:18.300555 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:18.302359 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:18.302888 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:18.302893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:18.302924 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:18.303019 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:18.303495 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:18.303498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:18.303514 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:18.303693 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:18.303769 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.304405 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.304501 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 09:44:18.305869 24911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.292548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:45045 (local address 127.3.147.2:59869)
0812 09:44:18.292993 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.293033 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:18.294218 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:18.294772 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:18.294778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:18.295135 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:18.295615 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.295627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.296522 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.296527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.297389 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.297398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.298029 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.298033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.298223 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.298858 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:18.298875 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:18.300356 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:18.302510 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:18.302514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:18.302525 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:18.302783 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:18.303188 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:18.303194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:18.303199 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:18.303390 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:18.303784 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:18.303793 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:18.304126 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 09:44:18.304363 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 09:44:18.304752 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":115,"threads_started":1}
I20260812 09:44:18.307116 24908 heartbeater.cc:347] Connected to a master server at 127.3.147.62:45045
I20260812 09:44:18.307420 24908 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:18.307902 24908 heartbeater.cc:511] Master 127.3.147.62:45045 requested a full tablet report, sending...
I20260812 09:44:18.308993 24550 ts_manager.cc:194] Registered new tserver with Master: 59c12c06fd484e96b91e2cd1f1677f01 (127.3.147.2:34395)
I20260812 09:44:18.309610 24550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:59869
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:44:18.327723 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:45045
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:18.439878 24915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:18.440147 24915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:18.440243 24915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:18.444298 24915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:18.444380 24915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:18.444470 24915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:44:18.449707 24915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:45045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:18.450783 24915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:18.451807 24915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:18.461443 24921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:18.461455 24925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:18.461452 24922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:18.461822 24915 server_base.cc:1034] running on GCE node
I20260812 09:44:18.462247 24915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:18.462841 24915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:18.464020 24915 hybrid_clock.cc:648] HybridClock initialized: now 1786527858463988 us; error 52 us; skew 500 ppm
Aug 12 09:44:18 dist-test-slave-cbbz krb5kdc[24499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527858, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:18.466920 24915 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:44:18.467775 24915 webserver.cc:460] Webserver started at http://127.3.147.3:40425/ using document root <none> and password file <none>
I20260812 09:44:18.468400 24915 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:18.468492 24915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:18.468720 24915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:18.470680 24915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "72158f3b4068498392cd536fdfd46d4d"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "4c752920d6e1e2b417d5a5ecc05389b6"
server_key_iv: "b40f688ce605a0e18c2d19f3f61f7e24"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.471235 24915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "72158f3b4068498392cd536fdfd46d4d"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "4c752920d6e1e2b417d5a5ecc05389b6"
server_key_iv: "b40f688ce605a0e18c2d19f3f61f7e24"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.475171 24915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 09:44:18.477619 24933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.478807 24915 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:44:18.478937 24915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "72158f3b4068498392cd536fdfd46d4d"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "4c752920d6e1e2b417d5a5ecc05389b6"
server_key_iv: "b40f688ce605a0e18c2d19f3f61f7e24"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.479046 24915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:18.524680 24915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:18.527791 24915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:18.528043 24915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:18.528690 24915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:18.529528 24915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:18.529600 24915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.529671 24915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:18.529722 24915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.539019 24915 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:33661
I20260812 09:44:18.539052 25062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:33661 every 8 connection(s)
I20260812 09:44:18.539853 24915 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:44:18.544580 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 24915
I20260812 09:44:18.544708 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:44:18.545042 3660 external_mini_cluster.cc:1428] Setting key 665f030afccbc89e3dff8fc6ea79a39c
Aug 12 09:44:18 dist-test-slave-cbbz krb5kdc[24499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527858, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:18.553444 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.541606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:40667 (local address 127.3.147.62:45045)
0812 09:44:18.541740 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.541744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.542604 (+ 860us) server_negotiation.cc:407] Connection header received
0812 09:44:18.543900 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.543904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.543962 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.544066 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:18.545599 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.546137 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.547289 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.547519 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.547625 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.550164 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:18.550167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:18.550169 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:18.550192 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:18.551608 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:18.552055 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:18.552057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:18.552059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:18.552225 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:18.552543 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:18.552545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:18.552546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:18.552732 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:18.552808 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.553232 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.553326 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:18.554291 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.542007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:45045 (local address 127.3.147.3:40667)
0812 09:44:18.542445 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.542488 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:18.543682 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:18.544264 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:18.544271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:18.544799 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:18.545455 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.545471 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.546267 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.546271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.547162 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.547171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.547676 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.547680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.547847 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.548423 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:18.548439 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:18.549983 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:18.551715 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:18.551719 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:18.551730 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:18.551966 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:18.552332 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:18.552334 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:18.552335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:18.552466 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:18.552819 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:18.552824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:18.553037 (+ 213us) client_negotiation.cc:769] Sending connection context
0812 09:44:18.553184 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 09:44:18.553431 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":113,"threads_started":1}
I20260812 09:44:18.555454 25063 heartbeater.cc:347] Connected to a master server at 127.3.147.62:45045
I20260812 09:44:18.555687 25063 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:18.556232 25063 heartbeater.cc:511] Master 127.3.147.62:45045 requested a full tablet report, sending...
I20260812 09:44:18.557332 24550 ts_manager.cc:194] Registered new tserver with Master: 72158f3b4068498392cd536fdfd46d4d (127.3.147.3:33661)
I20260812 09:44:18.558043 24550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:40667
I20260812 09:44:18.558957 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:44:18.569077 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.561118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32962 (local address 127.3.147.62:45045)
0812 09:44:18.561251 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.561255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.561503 (+ 248us) server_negotiation.cc:407] Connection header received
0812 09:44:18.561599 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.561602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.561638 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.561701 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:18.562442 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.562969 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.563929 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.564245 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.564347 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.566043 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:18.566046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:18.566048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:18.566068 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:18.567394 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:18.567874 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:18.567877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:18.567878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:18.567945 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:18.568236 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:18.568238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:18.568239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:18.568422 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:18.568536 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.568787 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.568888 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:18.571864 24550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:44:18.574859 24550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:44:18.589529 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.583553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37679 (local address 127.0.0.1:33682)
0812 09:44:18.584103 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.584117 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:18.584243 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:18.585227 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:18.585230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:18.585245 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:18.585310 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.585314 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.586424 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.586430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.588103 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.588109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.588954 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.588958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.589060 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.589096 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 09:44:18.589230 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 09:44:18.589381 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":319,"thread_start_us":134,"threads_started":1}
I20260812 09:44:18.589535 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.584012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.3.147.1:37679)
0812 09:44:18.584753 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.584783 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.584811 (+ 28us) server_negotiation.cc:407] Connection header received
0812 09:44:18.584893 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.584898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.585076 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.585471 (+ 395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:18.585618 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.586310 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.588263 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.588829 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.588954 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.589106 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.589274 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.589381 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":112,"threads_started":1}
I20260812 09:44:18.589777 25078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.583100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:34395 (local address 127.0.0.1:39066)
0812 09:44:18.583449 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.583500 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:18.583656 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:18.584802 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:18.584809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:18.584833 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:18.585014 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.585022 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.586540 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.586543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.587945 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.587954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.589134 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.589137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.589417 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.589442 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 09:44:18.589525 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 09:44:18.589595 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":310,"thread_start_us":87,"threads_started":1}
I20260812 09:44:18.590070 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.583867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:33661 (local address 127.0.0.1:33834)
0812 09:44:18.584404 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.584417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:18.584511 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:18.584992 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:18.584995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:18.585014 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:18.585116 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.585126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.586728 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.586731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.588081 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.588092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.589124 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.589128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.589232 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.589254 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 09:44:18.589305 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 09:44:18.589848 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":513,"thread_start_us":89,"threads_started":1}
I20260812 09:44:18.590090 25079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.583262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.3.147.2:34395)
0812 09:44:18.584244 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.584284 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.584313 (+ 29us) server_negotiation.cc:407] Connection header received
0812 09:44:18.584399 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.584404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.584616 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.585166 (+ 550us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:18.585330 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.586416 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.588068 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.588941 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.589216 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.589374 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.589571 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.589822 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":128,"threads_started":1}
I20260812 09:44:18.591089 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.584179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33834 (local address 127.3.147.3:33661)
0812 09:44:18.584541 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.584554 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.584576 (+ 22us) server_negotiation.cc:407] Connection header received
0812 09:44:18.584649 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.584654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.584831 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.585020 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:18.585201 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.586399 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.588224 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.588977 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.589982 (+ 1005us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.590132 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.590193 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.590301 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":153,"threads_started":1}
I20260812 09:44:18.592828 24685 tablet_service.cc:1467] Processing CreateTablet for tablet 038a99f15ab34166ba55f8e705ab0bb6 (DEFAULT_TABLE table=test-table [id=577f074e9f6d4fcd8462b9857a01bd6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:18.593139 24834 tablet_service.cc:1467] Processing CreateTablet for tablet 038a99f15ab34166ba55f8e705ab0bb6 (DEFAULT_TABLE table=test-table [id=577f074e9f6d4fcd8462b9857a01bd6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:18.593887 24685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038a99f15ab34166ba55f8e705ab0bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:18.593638 24988 tablet_service.cc:1467] Processing CreateTablet for tablet 038a99f15ab34166ba55f8e705ab0bb6 (DEFAULT_TABLE table=test-table [id=577f074e9f6d4fcd8462b9857a01bd6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:18.594175 24834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038a99f15ab34166ba55f8e705ab0bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:18.594549 24988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038a99f15ab34166ba55f8e705ab0bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:18.600198 25085 tablet_bootstrap.cc:492] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510: Bootstrap starting.
I20260812 09:44:18.601475 25087 tablet_bootstrap.cc:492] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: Bootstrap starting.
I20260812 09:44:18.602124 25086 tablet_bootstrap.cc:492] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: Bootstrap starting.
I20260812 09:44:18.602730 25085 tablet_bootstrap.cc:654] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:18.603618 25085 log.cc:824] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:18.604012 25086 tablet_bootstrap.cc:654] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:18.604326 25087 tablet_bootstrap.cc:654] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:18.605167 25087 log.cc:824] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:18.605244 25086 log.cc:824] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:18.606455 25085 tablet_bootstrap.cc:492] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510: No bootstrap required, opened a new log
I20260812 09:44:18.606683 25085 ts_tablet_manager.cc:1397] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 09:44:18.607131 25086 tablet_bootstrap.cc:492] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: No bootstrap required, opened a new log
I20260812 09:44:18.607138 25087 tablet_bootstrap.cc:492] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: No bootstrap required, opened a new log
I20260812 09:44:18.607311 25087 ts_tablet_manager.cc:1397] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260812 09:44:18.607311 25086 ts_tablet_manager.cc:1397] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:44:18.610106 25085 raft_consensus.cc:348] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.610359 25085 raft_consensus.cc:374] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:18.610414 25085 raft_consensus.cc:729] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7225bfd10144650a4d64f2e3074b510, State: Initialized, Role: FOLLOWER
I20260812 09:44:18.610859 25085 consensus_queue.cc:260] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.611126 25087 raft_consensus.cc:348] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.611126 25086 raft_consensus.cc:348] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.611423 25086 raft_consensus.cc:374] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:18.611423 25087 raft_consensus.cc:374] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:18.611505 25087 raft_consensus.cc:729] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72158f3b4068498392cd536fdfd46d4d, State: Initialized, Role: FOLLOWER
I20260812 09:44:18.611512 25086 raft_consensus.cc:729] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c12c06fd484e96b91e2cd1f1677f01, State: Initialized, Role: FOLLOWER
I20260812 09:44:18.611626 24758 heartbeater.cc:503] Master 127.3.147.62:45045 was elected leader, sending a full tablet report...
I20260812 09:44:18.611922 25086 consensus_queue.cc:260] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.611913 25087 consensus_queue.cc:260] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.612881 25087 ts_tablet_manager.cc:1428] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260812 09:44:18.612881 25086 ts_tablet_manager.cc:1428] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 09:44:18.612901 25085 ts_tablet_manager.cc:1428] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 09:44:18.613197 25063 heartbeater.cc:503] Master 127.3.147.62:45045 was elected leader, sending a full tablet report...
I20260812 09:44:18.614065 24908 heartbeater.cc:503] Master 127.3.147.62:45045 was elected leader, sending a full tablet report...
I20260812 09:44:18.672451 25091 raft_consensus.cc:482] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:18.672664 25091 raft_consensus.cc:504] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.673779 25091 leader_election.cc:288] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59c12c06fd484e96b91e2cd1f1677f01 (127.3.147.2:34395), 72158f3b4068498392cd536fdfd46d4d (127.3.147.3:33661)
I20260812 09:44:18.678864 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.674687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:56143 (local address 127.3.147.3:33661)
0812 09:44:18.674853 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.674859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.675128 (+ 269us) server_negotiation.cc:407] Connection header received
0812 09:44:18.675226 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.675229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.675278 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.675356 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:18.675617 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.676293 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.677633 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.678238 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.678353 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.678420 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.678674 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.678745 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 09:44:18.678891 25096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.674530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:33661 (local address 127.3.147.1:56143)
0812 09:44:18.675032 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.675045 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:18.675141 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:18.675372 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:18.675374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:18.675388 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:18.675506 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.675510 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.676395 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.676398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.677497 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.677508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.678368 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.678372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.678475 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.678562 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 09:44:18.678652 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 09:44:18.678725 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":41,"thread_start_us":102,"threads_started":1}
I20260812 09:44:18.678968 25095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.674081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:34395 (local address 127.3.147.1:48309)
0812 09:44:18.674395 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.674412 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:18.674529 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:18.674917 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:18.674920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:18.674954 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:18.675097 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.675107 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.676114 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.676116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.677450 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:18.677461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.678491 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.678495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:18.678630 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.678654 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 09:44:18.678731 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 09:44:18.678812 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":2,"thread_start_us":138,"threads_started":1}
I20260812 09:44:18.679333 25079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.674149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:48309 (local address 127.3.147.2:34395)
0812 09:44:18.674298 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.674302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.674511 (+ 209us) server_negotiation.cc:407] Connection header received
0812 09:44:18.674701 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.674706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.674770 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.674859 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:18.675219 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.676000 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.677601 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.678267 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.678361 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.678420 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.679150 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.679224 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:44:18.679508 25012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038a99f15ab34166ba55f8e705ab0bb6" candidate_uuid: "b7225bfd10144650a4d64f2e3074b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72158f3b4068498392cd536fdfd46d4d" is_pre_election: true
I20260812 09:44:18.679908 25012 raft_consensus.cc:2454] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7225bfd10144650a4d64f2e3074b510 in term 0.
I20260812 09:44:18.679935 24855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038a99f15ab34166ba55f8e705ab0bb6" candidate_uuid: "b7225bfd10144650a4d64f2e3074b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c12c06fd484e96b91e2cd1f1677f01" is_pre_election: true
I20260812 09:44:18.680219 24855 raft_consensus.cc:2454] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7225bfd10144650a4d64f2e3074b510 in term 0.
I20260812 09:44:18.680493 24634 leader_election.cc:302] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72158f3b4068498392cd536fdfd46d4d, b7225bfd10144650a4d64f2e3074b510; no voters:
I20260812 09:44:18.680817 25091 raft_consensus.cc:2781] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:44:18.680895 25091 raft_consensus.cc:482] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:44:18.680925 25091 raft_consensus.cc:3037] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:18.682062 25091 raft_consensus.cc:504] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.682439 25091 leader_election.cc:288] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [CANDIDATE]: Term 1 election: Requested vote from peers 59c12c06fd484e96b91e2cd1f1677f01 (127.3.147.2:34395), 72158f3b4068498392cd536fdfd46d4d (127.3.147.3:33661)
I20260812 09:44:18.682809 24855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038a99f15ab34166ba55f8e705ab0bb6" candidate_uuid: "b7225bfd10144650a4d64f2e3074b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c12c06fd484e96b91e2cd1f1677f01"
I20260812 09:44:18.682893 25012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038a99f15ab34166ba55f8e705ab0bb6" candidate_uuid: "b7225bfd10144650a4d64f2e3074b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72158f3b4068498392cd536fdfd46d4d"
I20260812 09:44:18.682932 24855 raft_consensus.cc:3037] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:18.683032 25012 raft_consensus.cc:3037] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:18.684333 25012 raft_consensus.cc:2454] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7225bfd10144650a4d64f2e3074b510 in term 1.
I20260812 09:44:18.684338 24855 raft_consensus.cc:2454] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7225bfd10144650a4d64f2e3074b510 in term 1.
I20260812 09:44:18.684649 24634 leader_election.cc:302] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72158f3b4068498392cd536fdfd46d4d, b7225bfd10144650a4d64f2e3074b510; no voters:
I20260812 09:44:18.684814 25091 raft_consensus.cc:2781] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:18.685027 25091 raft_consensus.cc:686] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [term 1 LEADER]: Becoming Leader. State: Replica: b7225bfd10144650a4d64f2e3074b510, State: Running, Role: LEADER
I20260812 09:44:18.685340 25091 consensus_queue.cc:237] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } }
I20260812 09:44:18.688421 24549 catalog_manager.cc:5495] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7225bfd10144650a4d64f2e3074b510 (127.3.147.1). New cstate: current_term: 1 leader_uuid: "b7225bfd10144650a4d64f2e3074b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7225bfd10144650a4d64f2e3074b510" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 37679 } health_report { overall_health: HEALTHY } } }
I20260812 09:44:18.711957 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.708408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.3.147.1:37679)
0812 09:44:18.708545 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.708550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.708585 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:44:18.708663 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.708671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.708724 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.708845 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:18.709096 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.709600 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.710451 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.710684 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.710819 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.711121 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:18.711516 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:18.711673 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.711777 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.711819 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:44:18.717818 24855 raft_consensus.cc:1261] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 1 FOLLOWER]: Refusing update from remote peer b7225bfd10144650a4d64f2e3074b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:18.717818 25012 raft_consensus.cc:1261] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 1 FOLLOWER]: Refusing update from remote peer b7225bfd10144650a4d64f2e3074b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:18.718533 25091 consensus_queue.cc:1035] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72158f3b4068498392cd536fdfd46d4d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 33661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:18.718726 25097 consensus_queue.cc:1035] T 038a99f15ab34166ba55f8e705ab0bb6 P b7225bfd10144650a4d64f2e3074b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59c12c06fd484e96b91e2cd1f1677f01" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:18.730175 25102 mvcc.cc:205] Tried to move back new op lower bound from 7317618109300670464 to 7317618109177630720. Current Snapshot: MvccSnapshot[applied={T|T < 7317618109300670464}]
I20260812 09:44:18.735517 24550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:18.735678 24550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:18.738130 24550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c214623cb45c41679824f6d7da2dc162: Sending DeleteTablet for 3 replicas of tablet 038a99f15ab34166ba55f8e705ab0bb6
I20260812 09:44:18.739177 24834 tablet_service.cc:1514] Processing DeleteTablet for tablet 038a99f15ab34166ba55f8e705ab0bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:18 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:39066
I20260812 09:44:18.739250 24988 tablet_service.cc:1514] Processing DeleteTablet for tablet 038a99f15ab34166ba55f8e705ab0bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:18 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:33834
I20260812 09:44:18.739250 24685 tablet_service.cc:1514] Processing DeleteTablet for tablet 038a99f15ab34166ba55f8e705ab0bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:18 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:33682
I20260812 09:44:18.739823 25110 tablet_replica.cc:330] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: stopping tablet replica
I20260812 09:44:18.740211 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 24617
I20260812 09:44:18.743163 25112 tablet_replica.cc:330] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: stopping tablet replica
I20260812 09:44:18.743456 25112 raft_consensus.cc:2229] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:18.743789 25112 raft_consensus.cc:2258] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:18.740252 25110 raft_consensus.cc:2229] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:18.744375 25110 raft_consensus.cc:2258] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:18.745795 25112 ts_tablet_manager.cc:1905] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:18.746239 25110 ts_tablet_manager.cc:1905] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:18.748826 25112 ts_tablet_manager.cc:1918] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:18.748941 25112 log.cc:1196] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/038a99f15ab34166ba55f8e705ab0bb6
I20260812 09:44:18.749028 25110 ts_tablet_manager.cc:1918] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:18.749123 25110 log.cc:1196] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/038a99f15ab34166ba55f8e705ab0bb6
I20260812 09:44:18.749377 25112 ts_tablet_manager.cc:1939] T 038a99f15ab34166ba55f8e705ab0bb6 P 72158f3b4068498392cd536fdfd46d4d: Deleting consensus metadata
I20260812 09:44:18.749430 25110 ts_tablet_manager.cc:1939] T 038a99f15ab34166ba55f8e705ab0bb6 P 59c12c06fd484e96b91e2cd1f1677f01: Deleting consensus metadata
I20260812 09:44:18.750650 24532 catalog_manager.cc:4841] TS 59c12c06fd484e96b91e2cd1f1677f01 (127.3.147.2:34395): tablet 038a99f15ab34166ba55f8e705ab0bb6 (table test-table [id=577f074e9f6d4fcd8462b9857a01bd6a]) successfully deleted
I20260812 09:44:18.750887 24532 catalog_manager.cc:4841] TS 72158f3b4068498392cd536fdfd46d4d (127.3.147.3:33661): tablet 038a99f15ab34166ba55f8e705ab0bb6 (table test-table [id=577f074e9f6d4fcd8462b9857a01bd6a]) successfully deleted
W20260812 09:44:18.750928 24533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:37679 (error 108)
I20260812 09:44:18.751271 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 24763
I20260812 09:44:18.753988 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.753698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:37679 (local address 127.0.0.1:33690)
0812 09:44:18.753835 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:18.753911 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:37679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 09:44:18.754237 24533 catalog_manager.cc:4584] TS b7225bfd10144650a4d64f2e3074b510 (127.3.147.1:37679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 038a99f15ab34166ba55f8e705ab0bb6: Network error: Client connection negotiation failed: client connection to 127.3.147.1:37679: connect: Connection refused (error 111)
I20260812 09:44:18.761145 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 24915
I20260812 09:44:18.770453 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 24515
2026-08-12T09:44:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2404 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T09:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:18Z Disabled control of system clock
I20260812 09:44:18.802834 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:37625
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:37625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 09:44:18.912216 25117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:18.912472 25117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:18.912508 25117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:18.916554 25117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:18.916610 25117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:18.916626 25117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:18.916643 25117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:18.916697 25117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:18.921487 25117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:37625
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:37625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:18.922577 25117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:18.923962 25117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:18.932734 25122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:18.932783 25125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:18.932714 25123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:18.932768 25117 server_base.cc:1034] running on GCE node
I20260812 09:44:18.933530 25117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:18.934641 25117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:18.935837 25117 hybrid_clock.cc:648] HybridClock initialized: now 1786527858935815 us; error 41 us; skew 500 ppm
I20260812 09:44:18.937719 25117 webserver.cc:460] Webserver started at http://127.3.147.62:44607/ using document root <none> and password file <none>
I20260812 09:44:18.938324 25117 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:18.938417 25117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:18.938645 25117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:18.940711 25117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "519019a31a114a93861a999099e257e2"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "a60f848a82053c3f04d9b8ec2f3dfcd7"
server_key_iv: "c187de084c69b3aec7d443eff6de60bd"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.941282 25117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "519019a31a114a93861a999099e257e2"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "a60f848a82053c3f04d9b8ec2f3dfcd7"
server_key_iv: "c187de084c69b3aec7d443eff6de60bd"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.944963 25117 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:44:18.947307 25132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:18.948635 25117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:18.948756 25117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "519019a31a114a93861a999099e257e2"
format_stamp: "Formatted at 2026-08-12 09:44:18 on dist-test-slave-cbbz"
server_key: "a60f848a82053c3f04d9b8ec2f3dfcd7"
server_key_iv: "c187de084c69b3aec7d443eff6de60bd"
server_key_version: "encryptionkey@0"
I20260812 09:44:18.948868 25117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:18.974375 25117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:18.975090 25117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:18.975302 25117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:18.982378 25117 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:37625
I20260812 09:44:18.982570 25188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:37625 every 8 connection(s)
I20260812 09:44:18.983399 25117 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:18.986578 25189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:18.988610 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 25117
I20260812 09:44:18.988708 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:18.989010 3660 external_mini_cluster.cc:1428] Setting key 8c25aea0a82f16152ef392c60517d6fd
I20260812 09:44:18.993062 25189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2: Bootstrap starting.
I20260812 09:44:18.995700 25189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:18.996526 25189 log.cc:824] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:18.998481 25189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2: No bootstrap required, opened a new log
I20260812 09:44:18.999289 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:18.990457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.3.147.62:37625)
0812 09:44:18.991156 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:18.991189 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:18.991236 (+ 47us) server_negotiation.cc:407] Connection header received
0812 09:44:18.992063 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:18.992099 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:18.992613 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:18.992973 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:18.993511 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.994992 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.995873 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:18.996400 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:18.996572 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:18.997487 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:18.997493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:18.997516 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:18.997896 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:18.997949 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:18.997970 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:18.998114 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:18.998193 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 09:44:18.998517 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":102,"threads_started":1}
I20260812 09:44:19.002455 25189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519019a31a114a93861a999099e257e2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37625 } }
I20260812 09:44:19.002694 25189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:19.002781 25189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519019a31a114a93861a999099e257e2, State: Initialized, Role: FOLLOWER
I20260812 09:44:19.003245 25189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519019a31a114a93861a999099e257e2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37625 } }
I20260812 09:44:19.003398 25189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:19.003487 25189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:19.003587 25189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:19.004590 25189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519019a31a114a93861a999099e257e2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37625 } }
I20260812 09:44:19.004920 25189 leader_election.cc:302] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 519019a31a114a93861a999099e257e2; no voters:
I20260812 09:44:19.005288 25189 leader_election.cc:288] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:19.005429 25196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:19.005659 25196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [term 1 LEADER]: Becoming Leader. State: Replica: 519019a31a114a93861a999099e257e2, State: Running, Role: LEADER
I20260812 09:44:19.005955 25196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519019a31a114a93861a999099e257e2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37625 } }
I20260812 09:44:19.006549 25189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:19.007552 25197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "519019a31a114a93861a999099e257e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519019a31a114a93861a999099e257e2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37625 } } }
I20260812 09:44:19.007689 25197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:19.007993 25198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 519019a31a114a93861a999099e257e2. Latest consensus state: current_term: 1 leader_uuid: "519019a31a114a93861a999099e257e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519019a31a114a93861a999099e257e2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37625 } } }
I20260812 09:44:19.008080 25198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:19.008211 25205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:19.010847 25205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:19.016388 25205 catalog_manager.cc:1347] Generated new cluster ID: ec54c5f44bbf4d678583c3838cc1dc1c
I20260812 09:44:19.016451 25205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:19.038990 25205 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:19.040021 25205 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:19.044790 25205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2: Generated new TSK 0
I20260812 09:44:19.045413 25205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:19.052323 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:37625
--builtin_ntp_servers=127.3.147.20:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 09:44:19.164513 25217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:19.164762 25217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:19.164799 25217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:19.169426 25217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:19.169478 25217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:19.169570 25217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:44:19.174281 25217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:19.175328 25217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:19.176848 25217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:19.186527 25223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:19.186493 25224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:19.186586 25226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:19.186623 25217 server_base.cc:1034] running on GCE node
I20260812 09:44:19.187142 25217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:19.187716 25217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:19.188902 25217 hybrid_clock.cc:648] HybridClock initialized: now 1786527859188878 us; error 35 us; skew 500 ppm
I20260812 09:44:19.190773 25217 webserver.cc:460] Webserver started at http://127.3.147.1:44281/ using document root <none> and password file <none>
I20260812 09:44:19.191372 25217 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:19.191463 25217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:19.191702 25217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:19.193881 25217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "3adeda1839ed46f2945c70e525eda5c8"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "a337ad580ae804514b5867b17d16ab41"
server_key_iv: "4d35ca9d2dbaf0fcc455791fc7598661"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.194447 25217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "3adeda1839ed46f2945c70e525eda5c8"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "a337ad580ae804514b5867b17d16ab41"
server_key_iv: "4d35ca9d2dbaf0fcc455791fc7598661"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.198079 25217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:44:19.200584 25232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.201838 25217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:19.201977 25217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "3adeda1839ed46f2945c70e525eda5c8"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "a337ad580ae804514b5867b17d16ab41"
server_key_iv: "4d35ca9d2dbaf0fcc455791fc7598661"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.202095 25217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:19.240427 25217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:19.241149 25217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:19.241359 25217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:19.241979 25217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:19.242931 25217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:19.243000 25217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.243072 25217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:19.243116 25217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.253209 25217 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:38195
I20260812 09:44:19.253285 25357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:38195 every 8 connection(s)
I20260812 09:44:19.254132 25217 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:44:19.259920 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 25217
I20260812 09:44:19.260047 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:44:19.260493 3660 external_mini_cluster.cc:1428] Setting key 891d877220c22e7b61724d9b573c816b
I20260812 09:44:19.265146 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.255914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:43279 (local address 127.3.147.62:37625)
0812 09:44:19.256086 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.256091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.257237 (+ 1146us) server_negotiation.cc:407] Connection header received
0812 09:44:19.258597 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.258600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.258647 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.258723 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:19.259870 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.260621 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.262447 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.262870 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.262993 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.264607 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:19.264611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:19.264614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:19.264630 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:19.264658 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:19.264669 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:19.264751 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.264943 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.265003 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 09:44:19.265940 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.256439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:37625 (local address 127.3.147.1:43279)
0812 09:44:19.257088 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.257129 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:19.258393 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:19.258884 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:19.258891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:19.258997 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:19.259692 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.259716 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.260759 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.260764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.262275 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.262291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.263106 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.263113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.263321 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.264055 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:19.264058 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:19.264107 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:19.264235 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:19.264773 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:19.264781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:19.264787 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 09:44:19.264906 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 09:44:19.265168 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":129,"threads_started":1}
I20260812 09:44:19.266928 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:37625
--builtin_ntp_servers=127.3.147.20:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 09:44:19.267124 25358 heartbeater.cc:347] Connected to a master server at 127.3.147.62:37625
I20260812 09:44:19.267361 25358 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:19.267866 25358 heartbeater.cc:511] Master 127.3.147.62:37625 requested a full tablet report, sending...
I20260812 09:44:19.270527 25148 ts_manager.cc:194] Registered new tserver with Master: 3adeda1839ed46f2945c70e525eda5c8 (127.3.147.1:38195)
I20260812 09:44:19.271628 25148 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.1:43279
W20260812 09:44:19.380446 25361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:19.380752 25361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:19.380827 25361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:19.384917 25361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:19.384995 25361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:19.385085 25361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:44:19.389818 25361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:19.390903 25361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:19.392355 25361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 09:44:19.402624 25361 server_base.cc:1034] running on GCE node
W20260812 09:44:19.402545 25367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:19.402577 25370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:19.402554 25368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:19.403288 25361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:19.403957 25361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:19.405149 25361 hybrid_clock.cc:648] HybridClock initialized: now 1786527859405142 us; error 59 us; skew 500 ppm
I20260812 09:44:19.406999 25361 webserver.cc:460] Webserver started at http://127.3.147.2:38325/ using document root <none> and password file <none>
I20260812 09:44:19.407668 25361 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:19.407733 25361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:19.407898 25361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:19.410033 25361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "bb0d00cb197cdea2e988c719c83b3964"
server_key_iv: "25ff67471d8d9a713dfd7d9707aa12eb"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.410564 25361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "bb0d00cb197cdea2e988c719c83b3964"
server_key_iv: "25ff67471d8d9a713dfd7d9707aa12eb"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.414253 25361 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:44:19.416651 25378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.417809 25361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:19.417946 25361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "bb0d00cb197cdea2e988c719c83b3964"
server_key_iv: "25ff67471d8d9a713dfd7d9707aa12eb"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.418045 25361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:19.448591 25361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:19.449327 25361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:19.449540 25361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:19.450224 25361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:19.451265 25361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:19.451311 25361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.451376 25361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:19.451409 25361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.460913 25361 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:32785
I20260812 09:44:19.460942 25499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:32785 every 8 connection(s)
I20260812 09:44:19.461845 25361 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:44:19.464798 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 25361
I20260812 09:44:19.464927 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:44:19.465324 3660 external_mini_cluster.cc:1428] Setting key 91272ae13356f488c3a2ed33e211134e
I20260812 09:44:19.467363 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:37625
--builtin_ntp_servers=127.3.147.20:44121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 09:44:19.471670 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.463728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:38355 (local address 127.3.147.62:37625)
0812 09:44:19.463897 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.463902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.465153 (+ 1251us) server_negotiation.cc:407] Connection header received
0812 09:44:19.466812 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.466815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.466867 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.466971 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:19.467957 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.468564 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.469722 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.469961 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.470071 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.471180 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:19.471183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:19.471186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:19.471204 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:19.471221 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:19.471229 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:19.471308 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.471467 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.471527 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 09:44:19.472563 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.464260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:37625 (local address 127.3.147.2:38355)
0812 09:44:19.465008 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.465058 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:19.466589 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:19.467150 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:19.467156 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:19.467242 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:19.467818 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.467832 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.468701 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.468704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.469584 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.469593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.470115 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.470119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.470279 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.470915 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:19.470916 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:19.470956 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:19.471017 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:19.471310 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:19.471323 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:19.471329 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 09:44:19.471453 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 09:44:19.471728 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":119,"threads_started":1}
I20260812 09:44:19.473690 25500 heartbeater.cc:347] Connected to a master server at 127.3.147.62:37625
I20260812 09:44:19.473945 25500 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:19.474455 25500 heartbeater.cc:511] Master 127.3.147.62:37625 requested a full tablet report, sending...
I20260812 09:44:19.475513 25148 ts_manager.cc:194] Registered new tserver with Master: 30f2c3eabc6c4dc7a07a7c287e92f1b7 (127.3.147.2:32785)
I20260812 09:44:19.476424 25148 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.2:38355
W20260812 09:44:19.583267 25503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:19.583501 25503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:19.583539 25503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:19.587833 25503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:19.587909 25503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:19.588060 25503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:44:19.593880 25503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:19.595065 25503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:19.596642 25503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:19.607367 25508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:19.607391 25509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:19.607367 25511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:19.607753 25503 server_base.cc:1034] running on GCE node
I20260812 09:44:19.608361 25503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:19.608989 25503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:19.610168 25503 hybrid_clock.cc:648] HybridClock initialized: now 1786527859610151 us; error 31 us; skew 500 ppm
I20260812 09:44:19.612017 25503 webserver.cc:460] Webserver started at http://127.3.147.3:42091/ using document root <none> and password file <none>
I20260812 09:44:19.612625 25503 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:19.612690 25503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:19.612896 25503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:19.614941 25503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "13f72a1837f9495c9da4174cadd5ecb9"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "a38f0350f3b0803faaea650240ab658b"
server_key_iv: "92cc85e742d227d25123c187309225d8"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.615497 25503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "13f72a1837f9495c9da4174cadd5ecb9"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "a38f0350f3b0803faaea650240ab658b"
server_key_iv: "92cc85e742d227d25123c187309225d8"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.619177 25503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 09:44:19.621647 25519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.622872 25503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:44:19.623001 25503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "13f72a1837f9495c9da4174cadd5ecb9"
format_stamp: "Formatted at 2026-08-12 09:44:19 on dist-test-slave-cbbz"
server_key: "a38f0350f3b0803faaea650240ab658b"
server_key_iv: "92cc85e742d227d25123c187309225d8"
server_key_version: "encryptionkey@0"
I20260812 09:44:19.623112 25503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:19.647042 25503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:19.647778 25503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:19.647989 25503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:19.648681 25503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:19.649681 25503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:19.649751 25503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.649820 25503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:19.649868 25503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:19.659008 25503 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:34085
I20260812 09:44:19.659047 25644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:34085 every 8 connection(s)
I20260812 09:44:19.659912 25503 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:44:19.664443 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 25503
I20260812 09:44:19.664567 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:44:19.664875 3660 external_mini_cluster.cc:1428] Setting key 89a5297ad99aaa1580c04f286a814fa1
I20260812 09:44:19.669449 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.661772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:40461 (local address 127.3.147.62:37625)
0812 09:44:19.661923 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.661927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.662985 (+ 1058us) server_negotiation.cc:407] Connection header received
0812 09:44:19.664401 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.664405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.664469 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.664572 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:19.665621 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.666379 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.667650 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.667866 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.667946 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.669034 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:19.669037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:19.669039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:19.669050 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:19.669068 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:19.669076 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:19.669134 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.669293 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.669341 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 09:44:19.670253 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.662177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:37625 (local address 127.3.147.3:40461)
0812 09:44:19.662830 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.662871 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:19.664183 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:19.664775 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:19.664782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:19.664864 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:19.665447 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.665466 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.666531 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.666533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.667539 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.667550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.668001 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.668004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.668219 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.668795 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:19.668797 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:19.668836 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:19.668897 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:19.669161 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:19.669168 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:19.669172 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 09:44:19.669264 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 09:44:19.669485 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260812 09:44:19.671371 25645 heartbeater.cc:347] Connected to a master server at 127.3.147.62:37625
I20260812 09:44:19.671600 25645 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:19.672092 25645 heartbeater.cc:511] Master 127.3.147.62:37625 requested a full tablet report, sending...
I20260812 09:44:19.673156 25148 ts_manager.cc:194] Registered new tserver with Master: 13f72a1837f9495c9da4174cadd5ecb9 (127.3.147.3:34085)
I20260812 09:44:19.673744 25148 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.3:40461
I20260812 09:44:19.678666 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:44:19.684748 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.680858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.3.147.62:37625)
0812 09:44:19.681004 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.681010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.681276 (+ 266us) server_negotiation.cc:407] Connection header received
0812 09:44:19.681387 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.681389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.681426 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.681525 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:19.681758 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.682288 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.683047 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.683267 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.683339 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.684238 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:19.684241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:19.684243 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:19.684255 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:19.684274 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:19.684281 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:19.684389 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.684523 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.684581 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 09:44:19.687438 25148 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:44:19.690466 25148 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:44:19.705014 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.698821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:34085 (local address 127.0.0.1:36692)
0812 09:44:19.700124 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.700162 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:19.700326 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:19.700996 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:19.700999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:19.701016 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:19.701106 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.701112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.702405 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.702409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.703351 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.703360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.704435 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.704438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.704541 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.704577 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 09:44:19.704718 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 09:44:19.704849 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1234,"mutex_wait_us":124,"thread_start_us":110,"threads_started":1}
I20260812 09:44:19.705030 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.699361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.3.147.3:34085)
0812 09:44:19.699961 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.699973 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.700251 (+ 278us) server_negotiation.cc:407] Connection header received
0812 09:44:19.700481 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.700486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.700685 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.700821 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:19.701233 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.702288 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.703496 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.704342 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.704437 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.704642 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.704776 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.704878 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":172,"threads_started":1}
I20260812 09:44:19.706187 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.698821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:38195 (local address 127.0.0.1:39840)
0812 09:44:19.699254 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.699308 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:19.699452 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:19.700455 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:19.700461 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:19.700485 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:19.700622 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.700629 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.702124 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.702128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.703564 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.703581 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.705521 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.705526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.705690 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.705819 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 09:44:19.705915 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 09:44:19.705997 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":56,"thread_start_us":105,"threads_started":1}
I20260812 09:44:19.706195 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.699918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.3.147.2:32785)
0812 09:44:19.700837 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.700859 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.700886 (+ 27us) server_negotiation.cc:407] Connection header received
0812 09:44:19.700971 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.700977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.701182 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.701354 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:19.701767 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.702973 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.704418 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.705304 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.705423 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.705699 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.705864 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.705974 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":178,"threads_started":1}
I20260812 09:44:19.706359 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.698966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.3.147.1:38195)
0812 09:44:19.699688 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.699723 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.699753 (+ 30us) server_negotiation.cc:407] Connection header received
0812 09:44:19.699843 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.699848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.700068 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.700269 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:19.700770 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.701981 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.703744 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.704681 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.704806 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.705019 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.706097 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.706201 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":105,"threads_started":1}
I20260812 09:44:19.706012 25663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.699713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:32785 (local address 127.0.0.1:58424)
0812 09:44:19.700319 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.700336 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:19.700446 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:19.701510 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:19.701513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:19.701530 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:19.701628 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.701633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.703114 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.703117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.704262 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.704274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.705515 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.705519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.705633 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.705724 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 09:44:19.705804 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 09:44:19.705872 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":61,"thread_start_us":118,"threads_started":1}
I20260812 09:44:19.708482 25573 tablet_service.cc:1467] Processing CreateTablet for tablet a8ee4bf00ee24ff08e8ef32fa138154a (DEFAULT_TABLE table=test-table [id=bb626b1694d64dfdbd9cab6953a247e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:19.709534 25573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ee4bf00ee24ff08e8ef32fa138154a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:19.709774 25430 tablet_service.cc:1467] Processing CreateTablet for tablet a8ee4bf00ee24ff08e8ef32fa138154a (DEFAULT_TABLE table=test-table [id=bb626b1694d64dfdbd9cab6953a247e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:19.710664 25430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ee4bf00ee24ff08e8ef32fa138154a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:19.716478 25665 tablet_bootstrap.cc:492] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Bootstrap starting.
I20260812 09:44:19.718494 25666 tablet_bootstrap.cc:492] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: Bootstrap starting.
I20260812 09:44:19.718402 25284 tablet_service.cc:1467] Processing CreateTablet for tablet a8ee4bf00ee24ff08e8ef32fa138154a (DEFAULT_TABLE table=test-table [id=bb626b1694d64dfdbd9cab6953a247e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:19.719244 25665 tablet_bootstrap.cc:654] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:19.719568 25284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ee4bf00ee24ff08e8ef32fa138154a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:19.720317 25665 log.cc:824] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:19.720534 25666 tablet_bootstrap.cc:654] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:19.721331 25666 log.cc:824] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:19.722826 25665 tablet_bootstrap.cc:492] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: No bootstrap required, opened a new log
I20260812 09:44:19.723033 25665 ts_tablet_manager.cc:1397] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260812 09:44:19.723471 25666 tablet_bootstrap.cc:492] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: No bootstrap required, opened a new log
I20260812 09:44:19.723718 25666 ts_tablet_manager.cc:1397] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:44:19.727278 25670 tablet_bootstrap.cc:492] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8: Bootstrap starting.
I20260812 09:44:19.727874 25665 raft_consensus.cc:348] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.728199 25665 raft_consensus.cc:374] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:19.728260 25665 raft_consensus.cc:729] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f72a1837f9495c9da4174cadd5ecb9, State: Initialized, Role: FOLLOWER
I20260812 09:44:19.728348 25666 raft_consensus.cc:348] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.728602 25666 raft_consensus.cc:374] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:19.728677 25666 raft_consensus.cc:729] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f2c3eabc6c4dc7a07a7c287e92f1b7, State: Initialized, Role: FOLLOWER
I20260812 09:44:19.728756 25665 consensus_queue.cc:260] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.729102 25666 consensus_queue.cc:260] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.729817 25665 ts_tablet_manager.cc:1428] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260812 09:44:19.729935 25666 ts_tablet_manager.cc:1428] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 09:44:19.730193 25645 heartbeater.cc:503] Master 127.3.147.62:37625 was elected leader, sending a full tablet report...
I20260812 09:44:19.730239 25500 heartbeater.cc:503] Master 127.3.147.62:37625 was elected leader, sending a full tablet report...
I20260812 09:44:19.731475 25670 tablet_bootstrap.cc:654] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:19.732581 25670 log.cc:824] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:19.735301 25670 tablet_bootstrap.cc:492] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8: No bootstrap required, opened a new log
I20260812 09:44:19.735549 25670 ts_tablet_manager.cc:1397] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 09:44:19.740007 25670 raft_consensus.cc:348] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.740341 25670 raft_consensus.cc:374] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:19.740430 25670 raft_consensus.cc:729] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3adeda1839ed46f2945c70e525eda5c8, State: Initialized, Role: FOLLOWER
I20260812 09:44:19.740948 25670 consensus_queue.cc:260] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.741673 25358 heartbeater.cc:503] Master 127.3.147.62:37625 was elected leader, sending a full tablet report...
I20260812 09:44:19.741994 25670 ts_tablet_manager.cc:1428] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
W20260812 09:44:19.755381 25359 tablet.cc:2369] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:19.844389 25671 raft_consensus.cc:482] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:19.844601 25671 raft_consensus.cc:504] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.845764 25671 leader_election.cc:288] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3adeda1839ed46f2945c70e525eda5c8 (127.3.147.1:38195), 30f2c3eabc6c4dc7a07a7c287e92f1b7 (127.3.147.2:32785)
I20260812 09:44:19.849685 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.846048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:38195 (local address 127.3.147.3:55847)
0812 09:44:19.846192 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.846210 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:19.846329 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:19.846730 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:19.846734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:19.846752 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:19.846880 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.846887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.847672 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.847677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.848499 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.848509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.849212 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.849216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.849320 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.849347 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 09:44:19.849452 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 09:44:19.849523 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 09:44:19.849685 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.846320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:55847 (local address 127.3.147.1:38195)
0812 09:44:19.846457 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.846462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.846477 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:44:19.846542 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.846546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.846604 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.846712 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:19.847019 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.847544 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.848636 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.849075 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.849211 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.849273 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.849471 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.849540 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 09:44:19.850099 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.846480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:34763 (local address 127.3.147.2:32785)
0812 09:44:19.846641 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.846647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.846906 (+ 259us) server_negotiation.cc:407] Connection header received
0812 09:44:19.847032 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.847040 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.847084 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.847169 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:19.847467 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.847967 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.849034 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.849448 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.849580 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.849641 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.849903 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.849983 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 09:44:19.850453 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.846373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:32785 (local address 127.3.147.3:34763)
0812 09:44:19.846809 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.846820 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:19.846937 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:19.847181 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:19.847184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:19.847200 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:19.847355 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.847360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.848077 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.848082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.848912 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:19.848921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.849598 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.849608 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:19.849709 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.849766 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 09:44:19.850247 (+ 481us) client_negotiation.cc:240] Negotiation successful
0812 09:44:19.850324 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":158,"threads_started":1}
I20260812 09:44:19.850428 25306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ee4bf00ee24ff08e8ef32fa138154a" candidate_uuid: "13f72a1837f9495c9da4174cadd5ecb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3adeda1839ed46f2945c70e525eda5c8" is_pre_election: true
I20260812 09:44:19.850775 25306 raft_consensus.cc:2454] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13f72a1837f9495c9da4174cadd5ecb9 in term 0.
I20260812 09:44:19.851197 25452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ee4bf00ee24ff08e8ef32fa138154a" candidate_uuid: "13f72a1837f9495c9da4174cadd5ecb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" is_pre_election: true
I20260812 09:44:19.851415 25523 leader_election.cc:302] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f72a1837f9495c9da4174cadd5ecb9, 3adeda1839ed46f2945c70e525eda5c8; no voters:
I20260812 09:44:19.851492 25452 raft_consensus.cc:2454] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13f72a1837f9495c9da4174cadd5ecb9 in term 0.
I20260812 09:44:19.851722 25671 raft_consensus.cc:2781] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:44:19.851824 25671 raft_consensus.cc:482] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:44:19.851868 25671 raft_consensus.cc:3037] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:19.853096 25671 raft_consensus.cc:504] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.853428 25671 leader_election.cc:288] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [CANDIDATE]: Term 1 election: Requested vote from peers 3adeda1839ed46f2945c70e525eda5c8 (127.3.147.1:38195), 30f2c3eabc6c4dc7a07a7c287e92f1b7 (127.3.147.2:32785)
I20260812 09:44:19.853835 25306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ee4bf00ee24ff08e8ef32fa138154a" candidate_uuid: "13f72a1837f9495c9da4174cadd5ecb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3adeda1839ed46f2945c70e525eda5c8"
I20260812 09:44:19.853962 25306 raft_consensus.cc:3037] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:19.853976 25452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ee4bf00ee24ff08e8ef32fa138154a" candidate_uuid: "13f72a1837f9495c9da4174cadd5ecb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7"
I20260812 09:44:19.854147 25452 raft_consensus.cc:3037] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:19.855119 25306 raft_consensus.cc:2454] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13f72a1837f9495c9da4174cadd5ecb9 in term 1.
I20260812 09:44:19.855293 25452 raft_consensus.cc:2454] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13f72a1837f9495c9da4174cadd5ecb9 in term 1.
I20260812 09:44:19.855432 25523 leader_election.cc:302] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f72a1837f9495c9da4174cadd5ecb9, 3adeda1839ed46f2945c70e525eda5c8; no voters:
I20260812 09:44:19.855639 25671 raft_consensus.cc:2781] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:19.855845 25671 raft_consensus.cc:686] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 1 LEADER]: Becoming Leader. State: Replica: 13f72a1837f9495c9da4174cadd5ecb9, State: Running, Role: LEADER
I20260812 09:44:19.856213 25671 consensus_queue.cc:237] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } }
I20260812 09:44:19.859259 25150 catalog_manager.cc:5495] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13f72a1837f9495c9da4174cadd5ecb9 (127.3.147.3). New cstate: current_term: 1 leader_uuid: "13f72a1837f9495c9da4174cadd5ecb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13f72a1837f9495c9da4174cadd5ecb9" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 } health_report { overall_health: UNKNOWN } } }
I20260812 09:44:19.897751 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.894457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.3.147.3:34085)
0812 09:44:19.894592 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:19.894596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:19.894632 (+ 36us) server_negotiation.cc:407] Connection header received
0812 09:44:19.894707 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:19.894709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:19.894766 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:19.894859 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:19.895093 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.895595 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.896443 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:19.896670 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:19.896752 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:19.897009 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:19.897400 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:19.897527 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:19.897585 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 09:44:19.897622 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:44:19.903614 25452 raft_consensus.cc:1261] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 1 FOLLOWER]: Refusing update from remote peer 13f72a1837f9495c9da4174cadd5ecb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:19.903626 25306 raft_consensus.cc:1261] T a8ee4bf00ee24ff08e8ef32fa138154a P 3adeda1839ed46f2945c70e525eda5c8 [term 1 FOLLOWER]: Refusing update from remote peer 13f72a1837f9495c9da4174cadd5ecb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:19.904369 25671 consensus_queue.cc:1035] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3adeda1839ed46f2945c70e525eda5c8" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:19.904564 25677 consensus_queue.cc:1035] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f2c3eabc6c4dc7a07a7c287e92f1b7" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 09:44:19.910763 25647 tablet.cc:2369] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:19.912904 25680 mvcc.cc:205] Tried to move back new op lower bound from 7317618114158108672 to 7317618113973800960. Current Snapshot: MvccSnapshot[applied={T|T < 7317618114158108672}]
I20260812 09:44:19.913378 25683 mvcc.cc:205] Tried to move back new op lower bound from 7317618114158108672 to 7317618113973800960. Current Snapshot: MvccSnapshot[applied={T|T < 7317618114158108672}]
I20260812 09:44:19.921730 25148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:19.921912 25148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:19.924386 25148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 519019a31a114a93861a999099e257e2: Sending DeleteTablet for 3 replicas of tablet a8ee4bf00ee24ff08e8ef32fa138154a
I20260812 09:44:19.925148 25573 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ee4bf00ee24ff08e8ef32fa138154a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:19 UTC) from {username='slave'} at 127.0.0.1:36692
I20260812 09:44:19.925168 25430 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ee4bf00ee24ff08e8ef32fa138154a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:19 UTC) from {username='slave'} at 127.0.0.1:58424
I20260812 09:44:19.925629 25692 tablet_replica.cc:330] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: stopping tablet replica
I20260812 09:44:19.925834 25693 tablet_replica.cc:330] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: stopping tablet replica
I20260812 09:44:19.925771 25284 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ee4bf00ee24ff08e8ef32fa138154a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:19 UTC) from {username='slave'} at 127.0.0.1:39840
I20260812 09:44:19.926138 25692 raft_consensus.cc:2229] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:19.926178 25693 raft_consensus.cc:2229] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:44:19.926554 25692 raft_consensus.cc:2258] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:19.926656 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 25217
I20260812 09:44:19.926560 25693 raft_consensus.cc:2258] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:19.931574 25693 ts_tablet_manager.cc:1905] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:19.932842 25692 ts_tablet_manager.cc:1905] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:19.934196 25693 ts_tablet_manager.cc:1918] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:19.934365 25693 log.cc:1196] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/a8ee4bf00ee24ff08e8ef32fa138154a
I20260812 09:44:19.934754 25693 ts_tablet_manager.cc:1939] T a8ee4bf00ee24ff08e8ef32fa138154a P 13f72a1837f9495c9da4174cadd5ecb9: Deleting consensus metadata
I20260812 09:44:19.935529 25692 ts_tablet_manager.cc:1918] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:19.935631 25692 log.cc:1196] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/a8ee4bf00ee24ff08e8ef32fa138154a
I20260812 09:44:19.935943 25692 ts_tablet_manager.cc:1939] T a8ee4bf00ee24ff08e8ef32fa138154a P 30f2c3eabc6c4dc7a07a7c287e92f1b7: Deleting consensus metadata
I20260812 09:44:19.936324 25133 catalog_manager.cc:4841] TS 13f72a1837f9495c9da4174cadd5ecb9 (127.3.147.3:34085): tablet a8ee4bf00ee24ff08e8ef32fa138154a (table test-table [id=bb626b1694d64dfdbd9cab6953a247e7]) successfully deleted
I20260812 09:44:19.937071 25136 catalog_manager.cc:4841] TS 30f2c3eabc6c4dc7a07a7c287e92f1b7 (127.3.147.2:32785): tablet a8ee4bf00ee24ff08e8ef32fa138154a (table test-table [id=bb626b1694d64dfdbd9cab6953a247e7]) successfully deleted
W20260812 09:44:19.940168 25136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:38195 (error 108)
I20260812 09:44:19.940337 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 25361
I20260812 09:44:19.941370 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:19.941026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:38195 (local address 127.0.0.1:39844)
0812 09:44:19.941210 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:19.941274 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:38195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260812 09:44:19.941532 25136 catalog_manager.cc:4584] TS 3adeda1839ed46f2945c70e525eda5c8 (127.3.147.1:38195): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8ee4bf00ee24ff08e8ef32fa138154a: Network error: Client connection negotiation failed: client connection to 127.3.147.1:38195: connect: Connection refused (error 111)
I20260812 09:44:19.948767 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 25503
I20260812 09:44:19.955658 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 25117
2026-08-12T09:44:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1184 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:20 dist-test-slave-cbbz krb5kdc[25699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:20 dist-test-slave-cbbz krb5kdc[25699](info): set up 2 sockets
Aug 12 09:44:20 dist-test-slave-cbbz krb5kdc[25699](info): commencing operation
krb5kdc: starting...
W20260812 09:44:21.071159 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:21 dist-test-slave-cbbz krb5kdc[25699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:21.192340 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:42199
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:42199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:21.302322 25713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:21.302547 25713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:21.302587 25713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:21.306463 25713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:21.306514 25713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:21.306533 25713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:21.306550 25713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:21.306565 25713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:21.311565 25713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:42199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:42199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:21.312495 25713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:21.313421 25713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:21.321728 25719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:21.321781 25713 server_base.cc:1034] running on GCE node
W20260812 09:44:21.321718 25722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:21.321734 25720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:21.322572 25713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:21.323709 25713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:21.324908 25713 hybrid_clock.cc:648] HybridClock initialized: now 1786527861324892 us; error 35 us; skew 500 ppm
Aug 12 09:44:21 dist-test-slave-cbbz krb5kdc[25699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527861, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:21.327538 25713 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:21.328454 25713 webserver.cc:460] Webserver started at http://127.3.147.62:43671/ using document root <none> and password file <none>
I20260812 09:44:21.329001 25713 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:21.329092 25713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:21.329324 25713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:21.331305 25713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "3852cb5144494f829c16ea949ae0f50e"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "36745cadfea7f78ee2c14e504389c463"
server_key_iv: "d90b88074f7b8c2d8d843dfc0ff114a3"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.331858 25713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "3852cb5144494f829c16ea949ae0f50e"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "36745cadfea7f78ee2c14e504389c463"
server_key_iv: "d90b88074f7b8c2d8d843dfc0ff114a3"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.335613 25713 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 09:44:21.338217 25731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:21.339401 25713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:21.339537 25713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "3852cb5144494f829c16ea949ae0f50e"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "36745cadfea7f78ee2c14e504389c463"
server_key_iv: "d90b88074f7b8c2d8d843dfc0ff114a3"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.339653 25713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:21.353363 25713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:21.356043 25713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:21.356284 25713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:21.363406 25713 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:42199
I20260812 09:44:21.363445 25789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:42199 every 8 connection(s)
I20260812 09:44:21.364338 25713 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:21.366959 25790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:21.368114 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 25713
I20260812 09:44:21.368311 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:21.368695 3660 external_mini_cluster.cc:1428] Setting key 1c5e7687d48ddda4c8eb647a69a3ee49
I20260812 09:44:21.373405 25790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e: Bootstrap starting.
I20260812 09:44:21.375689 25790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:21.376523 25790 log.cc:824] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:44:21 dist-test-slave-cbbz krb5kdc[25699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:21.378602 25790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e: No bootstrap required, opened a new log
I20260812 09:44:21.382483 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:21.370151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.3.147.62:42199)
0812 09:44:21.370573 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:21.370587 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:21.370620 (+ 33us) server_negotiation.cc:407] Connection header received
0812 09:44:21.371235 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:21.371251 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:21.371550 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:21.371880 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:21.372787 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.373794 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.374668 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.375105 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.375273 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:21.377973 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:21.377978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:21.377991 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:21.378016 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:21.379716 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:21.380186 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:21.380192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:21.380196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:21.380309 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:21.380606 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:21.380610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:21.380611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:21.381002 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:21.381205 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:21.381512 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 09:44:21.381791 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":139,"threads_started":1}
I20260812 09:44:21.382380 25790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3852cb5144494f829c16ea949ae0f50e" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42199 } }
I20260812 09:44:21.382624 25790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:21.382690 25790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3852cb5144494f829c16ea949ae0f50e, State: Initialized, Role: FOLLOWER
I20260812 09:44:21.383147 25790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3852cb5144494f829c16ea949ae0f50e" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42199 } }
I20260812 09:44:21.383296 25790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:21.383353 25790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:21.383443 25790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:21.384438 25790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3852cb5144494f829c16ea949ae0f50e" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42199 } }
I20260812 09:44:21.384840 25790 leader_election.cc:302] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3852cb5144494f829c16ea949ae0f50e; no voters:
I20260812 09:44:21.385205 25790 leader_election.cc:288] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:21.385284 25795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:21.385558 25795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [term 1 LEADER]: Becoming Leader. State: Replica: 3852cb5144494f829c16ea949ae0f50e, State: Running, Role: LEADER
I20260812 09:44:21.385869 25795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3852cb5144494f829c16ea949ae0f50e" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42199 } }
I20260812 09:44:21.386628 25790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:21.387713 25797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3852cb5144494f829c16ea949ae0f50e. Latest consensus state: current_term: 1 leader_uuid: "3852cb5144494f829c16ea949ae0f50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3852cb5144494f829c16ea949ae0f50e" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42199 } } }
I20260812 09:44:21.387861 25797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:21.387691 25796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3852cb5144494f829c16ea949ae0f50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3852cb5144494f829c16ea949ae0f50e" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 42199 } } }
I20260812 09:44:21.388126 25796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:21.388406 25807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:21.391388 25807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:21.397233 25807 catalog_manager.cc:1347] Generated new cluster ID: b10fb90f8b674c73bbf0f5e1b4a3c207
I20260812 09:44:21.397305 25807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:21.420267 25807 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:21.421773 25807 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:21.428557 25807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e: Generated new TSK 0
I20260812 09:44:21.429314 25807 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.147.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
Entry for principal kudu/127.3.147.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab.
I20260812 09:44:21.463698 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42199
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:21.577178 25820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:21.577400 25820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:21.577461 25820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:21.581420 25820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:21.581465 25820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:21.581517 25820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:44:21.586568 25820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.1.keytab
--principal=kudu/127.3.147.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:21.587692 25820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:21.588781 25820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:21.598105 25829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:21.598112 25825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:21.598127 25826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:21.598443 25820 server_base.cc:1034] running on GCE node
I20260812 09:44:21.598910 25820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:21.599509 25820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:21.600692 25820 hybrid_clock.cc:648] HybridClock initialized: now 1786527861600673 us; error 30 us; skew 500 ppm
Aug 12 09:44:21 dist-test-slave-cbbz krb5kdc[25699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527861, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:21.603381 25820 init.cc:377] Logged in from keytab as kudu/127.3.147.1@KRBTEST.COM (short username kudu)
I20260812 09:44:21.604243 25820 webserver.cc:460] Webserver started at http://127.3.147.1:43243/ using document root <none> and password file <none>
I20260812 09:44:21.604780 25820 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:21.604872 25820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:21.605154 25820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:21.607071 25820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "ad3b190cea63499ba9fc6f4e6a750fc4"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "3af87cef913245e8b79ce34f9ddc69a9"
server_key_iv: "b47d13f3093cc523ae97b2bc0ef1ab55"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.607631 25820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "ad3b190cea63499ba9fc6f4e6a750fc4"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "3af87cef913245e8b79ce34f9ddc69a9"
server_key_iv: "b47d13f3093cc523ae97b2bc0ef1ab55"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.611521 25820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 09:44:21.613987 25836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:21.615141 25820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 09:44:21.615271 25820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "ad3b190cea63499ba9fc6f4e6a750fc4"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "3af87cef913245e8b79ce34f9ddc69a9"
server_key_iv: "b47d13f3093cc523ae97b2bc0ef1ab55"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.615377 25820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:21.634631 25820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:21.637477 25820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:21.637670 25820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:21.638232 25820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:21.639110 25820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:21.639175 25820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:21.639236 25820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:21.639288 25820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:21.648788 25820 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:35009
I20260812 09:44:21.648816 25966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:35009 every 8 connection(s)
I20260812 09:44:21.649592 25820 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:44:21.650256 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 25820
I20260812 09:44:21.650400 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:44:21.650652 3660 external_mini_cluster.cc:1428] Setting key 10d256c5bb186fc29db6c965b7f64383
Aug 12 09:44:21 dist-test-slave-cbbz krb5kdc[25699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527861, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.1@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.2@KRBTEST.COM; defaulting to no policy
I20260812 09:44:21.663494 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:21.651412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:33445 (local address 127.3.147.62:42199)
0812 09:44:21.651539 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:21.651542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:21.652475 (+ 933us) server_negotiation.cc:407] Connection header received
0812 09:44:21.653751 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:21.653755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:21.653803 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:21.653958 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:21.655060 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.655687 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.656900 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.657136 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.657246 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:21.659837 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:21.659840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:21.659842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:21.659863 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:21.661237 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:21.661887 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:21.661892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:21.661894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:21.661975 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:21.662314 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:21.662319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:21.662321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:21.662492 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:21.662602 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:21.663189 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 09:44:21.663326 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.2@KRBTEST.COM" created.
I20260812 09:44:21.664394 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:21.651755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42199 (local address 127.3.147.1:33445)
0812 09:44:21.652340 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:21.652381 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:21.653546 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:21.654121 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:21.654129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:21.654483 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:21.654918 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:21.654931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.655856 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.655859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:21.656775 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:21.656785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.657284 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.657288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:21.657447 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:21.658022 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:21.658037 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:21.659673 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:21.661377 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:21.661383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:21.661396 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:21.661771 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:21.662070 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:21.662074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:21.662076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:21.662216 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:21.662610 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:21.662619 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:21.662953 (+ 334us) client_negotiation.cc:769] Sending connection context
0812 09:44:21.663146 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 09:44:21.663521 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":175,"threads_started":1}
I20260812 09:44:21.665745 25967 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42199
I20260812 09:44:21.665988 25967 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:21.666533 25967 heartbeater.cc:511] Master 127.3.147.62:42199 requested a full tablet report, sending...
I20260812 09:44:21.667995 25750 ts_manager.cc:194] Registered new tserver with Master: ad3b190cea63499ba9fc6f4e6a750fc4 (127.3.147.1:35009)
I20260812 09:44:21.669137 25750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.1@KRBTEST.COM'} at 127.3.147.1:33445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
Entry for principal kudu/127.3.147.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab.
I20260812 09:44:21.679147 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42199
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:21.792260 25972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:21.792505 25972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:21.792563 25972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:21.796547 25972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:21.796597 25972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:21.796653 25972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:44:21.801767 25972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.2.keytab
--principal=kudu/127.3.147.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:21.802685 25972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:21.803601 25972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:21.813028 25980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:21.813038 25981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:21.813119 25972 server_base.cc:1034] running on GCE node
W20260812 09:44:21.813326 25983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:21.813742 25972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:21.814317 25972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:21.815505 25972 hybrid_clock.cc:648] HybridClock initialized: now 1786527861815480 us; error 46 us; skew 500 ppm
Aug 12 09:44:21 dist-test-slave-cbbz krb5kdc[25699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527861, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:21.818372 25972 init.cc:377] Logged in from keytab as kudu/127.3.147.2@KRBTEST.COM (short username kudu)
I20260812 09:44:21.819242 25972 webserver.cc:460] Webserver started at http://127.3.147.2:34853/ using document root <none> and password file <none>
I20260812 09:44:21.819782 25972 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:21.819842 25972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:21.820000 25972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:21.822021 25972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "103153319c4446508bde3ceb10049711"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "cdde2c93ab7730e6b7080e4874d7eba7"
server_key_iv: "7c2b02ca58bb250591a44d7184ecb931"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.822515 25972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "103153319c4446508bde3ceb10049711"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "cdde2c93ab7730e6b7080e4874d7eba7"
server_key_iv: "7c2b02ca58bb250591a44d7184ecb931"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.826113 25972 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:44:21.828487 25992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:21.829663 25972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:44:21.829763 25972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "103153319c4446508bde3ceb10049711"
format_stamp: "Formatted at 2026-08-12 09:44:21 on dist-test-slave-cbbz"
server_key: "cdde2c93ab7730e6b7080e4874d7eba7"
server_key_iv: "7c2b02ca58bb250591a44d7184ecb931"
server_key_version: "encryptionkey@0"
I20260812 09:44:21.829844 25972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:21.840510 25972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:21.843185 25972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:21.843319 25972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:21.843858 25972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:21.844754 25972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:21.844800 25972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:21.844836 25972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:21.844851 25972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:21.853920 25972 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:36181
I20260812 09:44:21.853951 26123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:36181 every 8 connection(s)
I20260812 09:44:21.854751 25972 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:44:21.855300 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 25972
I20260812 09:44:21.855572 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:44:21.855901 3660 external_mini_cluster.cc:1428] Setting key e7f406b9815d1acc9d2224625efdc18d
Aug 12 09:44:21 dist-test-slave-cbbz krb5kdc[25699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527861, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.2@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.147.3@KRBTEST.COM; defaulting to no policy
I20260812 09:44:21.869982 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:21.856702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:42385 (local address 127.3.147.62:42199)
0812 09:44:21.856852 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:21.856856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:21.857813 (+ 957us) server_negotiation.cc:407] Connection header received
0812 09:44:21.859268 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:21.859275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:21.859337 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:21.859435 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:21.860976 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.861464 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.863338 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.863538 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.863614 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:21.866452 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:21.866455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:21.866457 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:21.866481 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:21.867963 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:21.868510 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:21.868512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:21.868514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:21.868586 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:21.868953 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:21.868956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:21.868957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:21.869145 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:21.869220 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:21.869655 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 09:44:21.869751 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.3@KRBTEST.COM" created.
I20260812 09:44:21.870802 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:21.857144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42199 (local address 127.3.147.2:42385)
0812 09:44:21.857649 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:21.857700 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:21.859036 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:21.859637 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:21.859649 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:21.860196 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:21.860850 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:21.860868 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.861598 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.861604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:21.863207 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:21.863219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:21.863675 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:21.863678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:21.863845 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:21.864465 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:21.864481 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:21.866278 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:21.868071 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:21.868076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:21.868086 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:21.868407 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:21.868718 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:21.868720 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:21.868721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:21.868857 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:21.869233 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:21.869238 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:21.869453 (+ 215us) client_negotiation.cc:769] Sending connection context
0812 09:44:21.869594 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 09:44:21.869845 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":151,"threads_started":1}
I20260812 09:44:21.872069 26124 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42199
I20260812 09:44:21.872334 26124 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:21.872819 26124 heartbeater.cc:511] Master 127.3.147.62:42199 requested a full tablet report, sending...
I20260812 09:44:21.873967 25750 ts_manager.cc:194] Registered new tserver with Master: 103153319c4446508bde3ceb10049711 (127.3.147.2:36181)
I20260812 09:44:21.874784 25750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.2@KRBTEST.COM'} at 127.3.147.2:42385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
Entry for principal kudu/127.3.147.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab.
I20260812 09:44:21.887959 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42199
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:22.000056 26129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:22.000370 26129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:22.000437 26129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:22.004418 26129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:22.004467 26129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:22.004570 26129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:44:22.009603 26129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.3.keytab
--principal=kudu/127.3.147.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:22.010706 26129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:22.011729 26129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:22.021137 26135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:22.021162 26137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:22.021251 26134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:22.021389 26129 server_base.cc:1034] running on GCE node
I20260812 09:44:22.021868 26129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:22.022459 26129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:22.023617 26129 hybrid_clock.cc:648] HybridClock initialized: now 1786527862023602 us; error 33 us; skew 500 ppm
Aug 12 09:44:22 dist-test-slave-cbbz krb5kdc[25699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527862, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:22.026262 26129 init.cc:377] Logged in from keytab as kudu/127.3.147.3@KRBTEST.COM (short username kudu)
I20260812 09:44:22.027119 26129 webserver.cc:460] Webserver started at http://127.3.147.3:46703/ using document root <none> and password file <none>
I20260812 09:44:22.027681 26129 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:22.027789 26129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:22.028018 26129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:22.030112 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "160dd91e1b0849528d931b2f4300ca0d"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "1de12181a8901ca51e9e690b94d683d0"
server_key_iv: "62e2e302770cd8666e559078bec1727a"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.030671 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "160dd91e1b0849528d931b2f4300ca0d"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "1de12181a8901ca51e9e690b94d683d0"
server_key_iv: "62e2e302770cd8666e559078bec1727a"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.034469 26129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:44:22.036969 26144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:22.038164 26129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:22.038300 26129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "160dd91e1b0849528d931b2f4300ca0d"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "1de12181a8901ca51e9e690b94d683d0"
server_key_iv: "62e2e302770cd8666e559078bec1727a"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.038399 26129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:22.066747 26129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:22.070201 26129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:22.070374 26129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:22.071043 26129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:22.071893 26129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:22.071940 26129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:22.072018 26129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:22.072053 26129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:22.081245 26129 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:45933
I20260812 09:44:22.081332 26272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:45933 every 8 connection(s)
I20260812 09:44:22.082190 26129 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:44:22.084801 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 26129
I20260812 09:44:22.084929 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:44:22.085296 3660 external_mini_cluster.cc:1428] Setting key 37cb0bab82ba368f34b44321befca9fa
Aug 12 09:44:22 dist-test-slave-cbbz krb5kdc[25699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527862, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.3@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:22.095574 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.083981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:33717 (local address 127.3.147.62:42199)
0812 09:44:22.084211 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.084215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.085171 (+ 956us) server_negotiation.cc:407] Connection header received
0812 09:44:22.086854 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.086857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.086921 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.087039 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:22.088440 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.088970 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.090030 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.090256 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.090335 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.092611 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:22.092614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:22.092616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:22.092638 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:22.093905 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:22.094340 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:22.094342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:22.094343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:22.094420 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:22.094695 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:22.094698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:22.094699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:22.094849 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:22.094920 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.095318 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.095442 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 09:44:22.096333 26275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.084454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:42199 (local address 127.3.147.3:33717)
0812 09:44:22.084998 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:22.085056 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:22.086619 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:22.087230 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:22.087236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:22.087699 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:22.088304 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.088338 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.089074 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.089076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.089923 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.089931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.090380 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.090382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.090508 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.090988 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:22.091003 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:22.092446 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:22.094006 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:22.094010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:22.094019 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:22.094250 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:22.094494 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:22.094496 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:22.094497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:22.094616 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:22.094930 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:22.094934 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:22.095124 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 09:44:22.095257 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 09:44:22.095517 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":148,"threads_started":1}
I20260812 09:44:22.097342 26273 heartbeater.cc:347] Connected to a master server at 127.3.147.62:42199
I20260812 09:44:22.097527 26273 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:22.097961 26273 heartbeater.cc:511] Master 127.3.147.62:42199 requested a full tablet report, sending...
I20260812 09:44:22.098981 25750 ts_manager.cc:194] Registered new tserver with Master: 160dd91e1b0849528d931b2f4300ca0d (127.3.147.3:45933)
I20260812 09:44:22.099452 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:44:22.099680 25750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.147.3@KRBTEST.COM'} at 127.3.147.3:33717
I20260812 09:44:22.107985 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.101570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.3.147.62:42199)
0812 09:44:22.101689 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.101693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.101951 (+ 258us) server_negotiation.cc:407] Connection header received
0812 09:44:22.102039 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.102041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.102075 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.102141 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:22.102756 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.103238 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.104017 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.104256 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.104330 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.105490 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:22.105492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:22.105494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:22.105514 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:22.106731 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:22.107110 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:22.107112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:22.107113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:22.107174 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:22.107379 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:22.107380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:22.107381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:22.107541 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:22.107631 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.107806 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.107883 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 09:44:22.110394 25750 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:44:22.113206 25750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:44:22.127249 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.121336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:45933 (local address 127.0.0.1:44314)
0812 09:44:22.121907 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:22.121922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:22.122008 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:22.122667 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:22.122673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:22.122694 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:22.122824 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.122832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.124208 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.124211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.125853 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.125864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.126713 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.126717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.126820 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.126855 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 09:44:22.126999 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 09:44:22.127111 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":182,"thread_start_us":100,"threads_started":1}
I20260812 09:44:22.127254 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.121653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.3.147.3:45933)
0812 09:44:22.122036 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.122051 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.122075 (+ 24us) server_negotiation.cc:407] Connection header received
0812 09:44:22.122144 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.122149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.122345 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.122493 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:22.122956 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.124052 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.125992 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.126584 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.126712 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.126849 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.127028 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.127112 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":159,"threads_started":1}
I20260812 09:44:22.127594 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.121117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:35009 (local address 127.0.0.1:41334)
0812 09:44:22.121436 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:22.121484 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:22.121611 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:22.122926 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:22.122929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:22.122948 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:22.123057 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.123064 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.124131 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.124155 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.125996 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.126004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.127042 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.127047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.127213 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.127271 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 09:44:22.127337 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 09:44:22.127444 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":417,"thread_start_us":72,"threads_started":1}
I20260812 09:44:22.128342 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.122075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.3.147.1:35009)
0812 09:44:22.122493 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.122508 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.122533 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:44:22.122608 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.122614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.122786 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.122948 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:22.123175 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.124005 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.126107 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.126912 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.127838 (+ 926us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.127976 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.128028 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.128126 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":149,"threads_started":1}
I20260812 09:44:22.128594 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.122241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56450 (local address 127.3.147.2:36181)
0812 09:44:22.122618 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.122647 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.123030 (+ 383us) server_negotiation.cc:407] Connection header received
0812 09:44:22.123110 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.123115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.123292 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.123434 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:22.123658 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.124576 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.125948 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.126754 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.126865 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.127021 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.128353 (+ 1332us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.128454 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":94,"threads_started":1}
I20260812 09:44:22.128767 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.121795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:36181 (local address 127.0.0.1:56450)
0812 09:44:22.122811 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:22.122824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:22.122918 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:22.123440 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:22.123443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:22.123462 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:22.123537 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.123546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.124705 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.124709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.125827 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.125837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.127074 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.127077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.127232 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.127281 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 09:44:22.127756 (+ 475us) client_negotiation.cc:240] Negotiation successful
0812 09:44:22.127830 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":87,"thread_start_us":83,"threads_started":1}
I20260812 09:44:22.130338 26201 tablet_service.cc:1467] Processing CreateTablet for tablet 72cc6bc4a45b4abfb5e26b1557893c1f (DEFAULT_TABLE table=test-table [id=4c6775cc3dc748b9baf2e7a9bc2e675d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:22.130605 25892 tablet_service.cc:1467] Processing CreateTablet for tablet 72cc6bc4a45b4abfb5e26b1557893c1f (DEFAULT_TABLE table=test-table [id=4c6775cc3dc748b9baf2e7a9bc2e675d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:22.131361 25892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72cc6bc4a45b4abfb5e26b1557893c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:22.131392 26201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72cc6bc4a45b4abfb5e26b1557893c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:22.131278 26046 tablet_service.cc:1467] Processing CreateTablet for tablet 72cc6bc4a45b4abfb5e26b1557893c1f (DEFAULT_TABLE table=test-table [id=4c6775cc3dc748b9baf2e7a9bc2e675d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:22.132119 26046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72cc6bc4a45b4abfb5e26b1557893c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:22.138079 26295 tablet_bootstrap.cc:492] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4: Bootstrap starting.
I20260812 09:44:22.138667 26297 tablet_bootstrap.cc:492] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: Bootstrap starting.
I20260812 09:44:22.140445 26296 tablet_bootstrap.cc:492] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Bootstrap starting.
I20260812 09:44:22.140688 26295 tablet_bootstrap.cc:654] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:22.141165 26297 tablet_bootstrap.cc:654] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:22.142058 26295 log.cc:824] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:22.142058 26297 log.cc:824] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:22.142457 26296 tablet_bootstrap.cc:654] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:22.143611 26296 log.cc:824] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:22.144017 26295 tablet_bootstrap.cc:492] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4: No bootstrap required, opened a new log
I20260812 09:44:22.144017 26297 tablet_bootstrap.cc:492] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: No bootstrap required, opened a new log
I20260812 09:44:22.144227 26297 ts_tablet_manager.cc:1397] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 09:44:22.144546 26295 ts_tablet_manager.cc:1397] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 09:44:22.145418 26296 tablet_bootstrap.cc:492] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: No bootstrap required, opened a new log
I20260812 09:44:22.145637 26296 ts_tablet_manager.cc:1397] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 09:44:22.148175 26297 raft_consensus.cc:348] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.148454 26297 raft_consensus.cc:374] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:22.148175 26295 raft_consensus.cc:348] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.148319 26296 raft_consensus.cc:348] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.148522 26297 raft_consensus.cc:729] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103153319c4446508bde3ceb10049711, State: Initialized, Role: FOLLOWER
I20260812 09:44:22.148542 26295 raft_consensus.cc:374] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:22.148583 26296 raft_consensus.cc:374] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:22.148597 26295 raft_consensus.cc:729] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad3b190cea63499ba9fc6f4e6a750fc4, State: Initialized, Role: FOLLOWER
I20260812 09:44:22.148650 26296 raft_consensus.cc:729] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160dd91e1b0849528d931b2f4300ca0d, State: Initialized, Role: FOLLOWER
I20260812 09:44:22.148995 26295 consensus_queue.cc:260] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.148995 26297 consensus_queue.cc:260] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.149034 26296 consensus_queue.cc:260] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.149631 26124 heartbeater.cc:503] Master 127.3.147.62:42199 was elected leader, sending a full tablet report...
I20260812 09:44:22.149963 26296 ts_tablet_manager.cc:1428] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 09:44:22.149986 26295 ts_tablet_manager.cc:1428] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 09:44:22.150312 25967 heartbeater.cc:503] Master 127.3.147.62:42199 was elected leader, sending a full tablet report...
I20260812 09:44:22.150714 26273 heartbeater.cc:503] Master 127.3.147.62:42199 was elected leader, sending a full tablet report...
I20260812 09:44:22.150795 26297 ts_tablet_manager.cc:1428] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260812 09:44:22.151641 25968 tablet.cc:2369] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:22.281647 26304 raft_consensus.cc:482] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:22.281857 26304 raft_consensus.cc:504] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.283034 26304 leader_election.cc:288] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad3b190cea63499ba9fc6f4e6a750fc4 (127.3.147.1:35009), 103153319c4446508bde3ceb10049711 (127.3.147.2:36181)
I20260812 09:44:22.287163 26275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.283246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:35009 (local address 127.3.147.3:59237)
0812 09:44:22.283397 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:22.283414 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:22.283518 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:22.283878 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:22.283881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:22.283901 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:22.284116 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.284124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.285020 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.285024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.285983 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.285993 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.286645 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.286652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.286763 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.286801 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 09:44:22.286874 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 09:44:22.286980 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":63,"spinlock_wait_cycles":8576}
I20260812 09:44:22.287168 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.283419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:59237 (local address 127.3.147.1:35009)
0812 09:44:22.283551 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.283557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.283588 (+ 31us) server_negotiation.cc:407] Connection header received
0812 09:44:22.283660 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.283662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.283734 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.283835 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:22.284300 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.284914 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.286115 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.286522 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.286603 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.286702 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.286925 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.287017 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 09:44:22.287317 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.283675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:53237 (local address 127.3.147.2:36181)
0812 09:44:22.283841 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.283847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.283957 (+ 110us) server_negotiation.cc:407] Connection header received
0812 09:44:22.284106 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.284108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.284183 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.284262 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:22.284560 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.285091 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.286163 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.286554 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.286711 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.286772 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.287089 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.287190 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 09:44:22.287534 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.283513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:36181 (local address 127.3.147.3:53237)
0812 09:44:22.283848 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:22.283861 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:22.284002 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:22.284304 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:22.284307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:22.284324 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:22.284446 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.284453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.285201 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.285203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.286004 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.286013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.286770 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.286773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.286907 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.286931 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 09:44:22.287064 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 09:44:22.287127 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":1,"spinlock_wait_cycles":27392,"thread_start_us":122,"threads_started":1}
I20260812 09:44:22.288002 25915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72cc6bc4a45b4abfb5e26b1557893c1f" candidate_uuid: "160dd91e1b0849528d931b2f4300ca0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" is_pre_election: true
I20260812 09:44:22.288254 26071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72cc6bc4a45b4abfb5e26b1557893c1f" candidate_uuid: "160dd91e1b0849528d931b2f4300ca0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103153319c4446508bde3ceb10049711" is_pre_election: true
I20260812 09:44:22.288342 25915 raft_consensus.cc:2454] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 160dd91e1b0849528d931b2f4300ca0d in term 0.
I20260812 09:44:22.288538 26071 raft_consensus.cc:2454] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 160dd91e1b0849528d931b2f4300ca0d in term 0.
I20260812 09:44:22.288841 26148 leader_election.cc:302] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160dd91e1b0849528d931b2f4300ca0d, ad3b190cea63499ba9fc6f4e6a750fc4; no voters:
I20260812 09:44:22.289083 26304 raft_consensus.cc:2781] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:44:22.289198 26304 raft_consensus.cc:482] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:44:22.289263 26304 raft_consensus.cc:3037] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:22.290392 26304 raft_consensus.cc:504] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.290735 26304 leader_election.cc:288] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [CANDIDATE]: Term 1 election: Requested vote from peers ad3b190cea63499ba9fc6f4e6a750fc4 (127.3.147.1:35009), 103153319c4446508bde3ceb10049711 (127.3.147.2:36181)
I20260812 09:44:22.291111 25915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72cc6bc4a45b4abfb5e26b1557893c1f" candidate_uuid: "160dd91e1b0849528d931b2f4300ca0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4"
I20260812 09:44:22.291201 26071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72cc6bc4a45b4abfb5e26b1557893c1f" candidate_uuid: "160dd91e1b0849528d931b2f4300ca0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103153319c4446508bde3ceb10049711"
I20260812 09:44:22.291246 25915 raft_consensus.cc:3037] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:22.291333 26071 raft_consensus.cc:3037] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:22.292587 26071 raft_consensus.cc:2454] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 160dd91e1b0849528d931b2f4300ca0d in term 1.
I20260812 09:44:22.292587 25915 raft_consensus.cc:2454] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 160dd91e1b0849528d931b2f4300ca0d in term 1.
I20260812 09:44:22.292937 26148 leader_election.cc:302] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103153319c4446508bde3ceb10049711, 160dd91e1b0849528d931b2f4300ca0d; no voters:
I20260812 09:44:22.293138 26304 raft_consensus.cc:2781] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:22.293387 26304 raft_consensus.cc:686] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 1 LEADER]: Becoming Leader. State: Replica: 160dd91e1b0849528d931b2f4300ca0d, State: Running, Role: LEADER
I20260812 09:44:22.293737 26304 consensus_queue.cc:237] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } }
I20260812 09:44:22.296904 25748 catalog_manager.cc:5495] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 160dd91e1b0849528d931b2f4300ca0d (127.3.147.3). New cstate: current_term: 1 leader_uuid: "160dd91e1b0849528d931b2f4300ca0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "160dd91e1b0849528d931b2f4300ca0d" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 45933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 } health_report { overall_health: UNKNOWN } } }
I20260812 09:44:22.319984 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.316603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.3.147.3:45933)
0812 09:44:22.316730 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.316734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.316748 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:44:22.316815 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.316817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.316865 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.316955 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:22.317207 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.317717 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.318585 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.318799 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.318881 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.319202 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:22.319667 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:22.319757 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.319834 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.319884 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:22.325537 25915 raft_consensus.cc:1261] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 1 FOLLOWER]: Refusing update from remote peer 160dd91e1b0849528d931b2f4300ca0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:22.325574 26071 raft_consensus.cc:1261] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 1 FOLLOWER]: Refusing update from remote peer 160dd91e1b0849528d931b2f4300ca0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:22.326231 26306 consensus_queue.cc:1035] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "103153319c4446508bde3ceb10049711" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:22.326412 26304 consensus_queue.cc:1035] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad3b190cea63499ba9fc6f4e6a750fc4" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 35009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 09:44:22.333112 26274 tablet.cc:2369] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:22.335048 26311 mvcc.cc:205] Tried to move back new op lower bound from 7317618124078731264 to 7317618123958001664. Current Snapshot: MvccSnapshot[applied={T|T < 7317618124078731264}]
I20260812 09:44:22.342847 25749 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:22.343007 25749 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:22.345315 25749 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3852cb5144494f829c16ea949ae0f50e: Sending DeleteTablet for 3 replicas of tablet 72cc6bc4a45b4abfb5e26b1557893c1f
I20260812 09:44:22.346077 26201 tablet_service.cc:1514] Processing DeleteTablet for tablet 72cc6bc4a45b4abfb5e26b1557893c1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:22 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:44314
I20260812 09:44:22.346244 26046 tablet_service.cc:1514] Processing DeleteTablet for tablet 72cc6bc4a45b4abfb5e26b1557893c1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:22 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:56450
I20260812 09:44:22.346266 25892 tablet_service.cc:1514] Processing DeleteTablet for tablet 72cc6bc4a45b4abfb5e26b1557893c1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:22 UTC) from {username='kudu', principal='kudu/127.3.147.62@KRBTEST.COM'} at 127.0.0.1:41334
I20260812 09:44:22.346577 26322 tablet_replica.cc:330] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: stopping tablet replica
I20260812 09:44:22.346732 26323 tablet_replica.cc:330] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: stopping tablet replica
I20260812 09:44:22.346727 26325 tablet_replica.cc:330] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4: stopping tablet replica
I20260812 09:44:22.346988 26322 raft_consensus.cc:2229] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:44:22.346987 26323 raft_consensus.cc:2229] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:22.347018 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 25820
I20260812 09:44:22.347016 26325 raft_consensus.cc:2229] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:22.347251 26323 raft_consensus.cc:2258] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:22.347268 26325 raft_consensus.cc:2258] T 72cc6bc4a45b4abfb5e26b1557893c1f P ad3b190cea63499ba9fc6f4e6a750fc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:22.347455 26322 raft_consensus.cc:2258] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:22.348693 26323 ts_tablet_manager.cc:1905] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:22.348693 26322 ts_tablet_manager.cc:1905] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:22.351305 26323 ts_tablet_manager.cc:1918] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:22.351305 26322 ts_tablet_manager.cc:1918] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:22.351395 26322 log.cc:1196] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/72cc6bc4a45b4abfb5e26b1557893c1f
I20260812 09:44:22.351503 26323 log.cc:1196] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/72cc6bc4a45b4abfb5e26b1557893c1f
I20260812 09:44:22.351729 26322 ts_tablet_manager.cc:1939] T 72cc6bc4a45b4abfb5e26b1557893c1f P 160dd91e1b0849528d931b2f4300ca0d: Deleting consensus metadata
I20260812 09:44:22.351897 26323 ts_tablet_manager.cc:1939] T 72cc6bc4a45b4abfb5e26b1557893c1f P 103153319c4446508bde3ceb10049711: Deleting consensus metadata
I20260812 09:44:22.352855 25734 catalog_manager.cc:4841] TS 160dd91e1b0849528d931b2f4300ca0d (127.3.147.3:45933): tablet 72cc6bc4a45b4abfb5e26b1557893c1f (table test-table [id=4c6775cc3dc748b9baf2e7a9bc2e675d]) successfully deleted
I20260812 09:44:22.353076 25735 catalog_manager.cc:4841] TS 103153319c4446508bde3ceb10049711 (127.3.147.2:36181): tablet 72cc6bc4a45b4abfb5e26b1557893c1f (table test-table [id=4c6775cc3dc748b9baf2e7a9bc2e675d]) successfully deleted
W20260812 09:44:22.358433 25735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:35009 (error 108)
I20260812 09:44:22.358700 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 25972
I20260812 09:44:22.359517 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.359213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:35009 (local address 127.0.0.1:41336)
0812 09:44:22.359372 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:22.359437 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:35009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 09:44:22.359668 25735 catalog_manager.cc:4584] TS ad3b190cea63499ba9fc6f4e6a750fc4 (127.3.147.1:35009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72cc6bc4a45b4abfb5e26b1557893c1f: Network error: Client connection negotiation failed: client connection to 127.3.147.1:35009: connect: Connection refused (error 111)
I20260812 09:44:22.369334 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 26129
I20260812 09:44:22.378726 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 25713
2026-08-12T09:44:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2422 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T09:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:22Z Disabled control of system clock
I20260812 09:44:22.410418 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:40431
--webserver_interface=127.3.147.62
--webserver_port=0
--builtin_ntp_servers=127.3.147.20:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:40431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 09:44:22.518043 26332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:22.518280 26332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:22.518314 26332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:22.522140 26332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:22.522186 26332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:22.522200 26332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:22.522217 26332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:22.522231 26332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:22.526726 26332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:40431
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:40431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:22.527643 26332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:22.528928 26332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:22.537201 26344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:22.537281 26341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:22.537375 26332 server_base.cc:1034] running on GCE node
W20260812 09:44:22.537191 26342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:22.538004 26332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:22.539049 26332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:22.540213 26332 hybrid_clock.cc:648] HybridClock initialized: now 1786527862540182 us; error 44 us; skew 500 ppm
I20260812 09:44:22.542008 26332 webserver.cc:460] Webserver started at http://127.3.147.62:34515/ using document root <none> and password file <none>
I20260812 09:44:22.542589 26332 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:22.542682 26332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:22.542920 26332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:22.545047 26332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "671d0cc603b5482d98682a79d932defd"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "e0b183e9a29dedb06c948d84f3c2c8de"
server_key_iv: "25e819278d76aee016e6173925e44e3e"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.545598 26332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "671d0cc603b5482d98682a79d932defd"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "e0b183e9a29dedb06c948d84f3c2c8de"
server_key_iv: "25e819278d76aee016e6173925e44e3e"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.549139 26332 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:22.551808 26350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:22.553061 26332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:22.553200 26332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "671d0cc603b5482d98682a79d932defd"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "e0b183e9a29dedb06c948d84f3c2c8de"
server_key_iv: "25e819278d76aee016e6173925e44e3e"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.553310 26332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:22.568274 26332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:22.568943 26332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:22.569140 26332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:22.576443 26332 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:40431
I20260812 09:44:22.576463 26412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:40431 every 8 connection(s)
I20260812 09:44:22.577397 26332 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:22.580554 26413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:22.585685 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 26332
I20260812 09:44:22.585798 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:22.586115 3660 external_mini_cluster.cc:1428] Setting key ca9ba9c388b7c79a46bea7aed9e8e2f4
I20260812 09:44:22.586273 26413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd: Bootstrap starting.
I20260812 09:44:22.589154 26413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:22.590034 26413 log.cc:824] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:22.592365 26413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd: No bootstrap required, opened a new log
I20260812 09:44:22.597106 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.587566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.3.147.62:40431)
0812 09:44:22.588309 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.588336 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.588380 (+ 44us) server_negotiation.cc:407] Connection header received
0812 09:44:22.589305 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.589331 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.589871 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.590264 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:22.590693 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.592188 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.593089 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.593768 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.593935 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.594723 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:22.594730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:22.594746 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:22.595255 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:22.595327 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:22.595353 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:22.595470 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.595564 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.596019 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":132,"threads_started":1}
I20260812 09:44:22.597106 26413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d0cc603b5482d98682a79d932defd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40431 } }
I20260812 09:44:22.597515 26413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:22.597617 26413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d0cc603b5482d98682a79d932defd, State: Initialized, Role: FOLLOWER
I20260812 09:44:22.598075 26413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d0cc603b5482d98682a79d932defd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40431 } }
I20260812 09:44:22.598237 26413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:22.598316 26413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:22.598448 26413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:22.599485 26413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d0cc603b5482d98682a79d932defd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40431 } }
I20260812 09:44:22.599869 26413 leader_election.cc:302] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 671d0cc603b5482d98682a79d932defd; no voters:
I20260812 09:44:22.600284 26413 leader_election.cc:288] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:22.600363 26420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:22.600623 26420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [term 1 LEADER]: Becoming Leader. State: Replica: 671d0cc603b5482d98682a79d932defd, State: Running, Role: LEADER
I20260812 09:44:22.600944 26420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d0cc603b5482d98682a79d932defd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40431 } }
I20260812 09:44:22.601574 26413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:22.602603 26422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 671d0cc603b5482d98682a79d932defd. Latest consensus state: current_term: 1 leader_uuid: "671d0cc603b5482d98682a79d932defd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d0cc603b5482d98682a79d932defd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40431 } } }
I20260812 09:44:22.602743 26422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:22.603057 26421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "671d0cc603b5482d98682a79d932defd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671d0cc603b5482d98682a79d932defd" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 40431 } } }
I20260812 09:44:22.603142 26430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:22.603152 26421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:22.606958 26430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:22.613613 26430 catalog_manager.cc:1347] Generated new cluster ID: 0d44a9ab524e4baa8dd41752f16a3809
I20260812 09:44:22.613696 26430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:22.637909 26430 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:22.639124 26430 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:22.653653 26430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd: Generated new TSK 0
I20260812 09:44:22.654505 26430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:22.658583 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.1:0
--local_ip_for_outbound_sockets=127.3.147.1
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40431
--builtin_ntp_servers=127.3.147.20:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 09:44:22.775213 26440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:22.775484 26440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:22.775568 26440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:22.779613 26440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:22.779670 26440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:22.779736 26440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.1
I20260812 09:44:22.784529 26440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.147.1
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.1
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:22.785620 26440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:22.786981 26440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:22.796633 26450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:22.796661 26447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:22.796806 26440 server_base.cc:1034] running on GCE node
W20260812 09:44:22.796633 26448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:22.797343 26440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:22.798014 26440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:22.799190 26440 hybrid_clock.cc:648] HybridClock initialized: now 1786527862799169 us; error 31 us; skew 500 ppm
I20260812 09:44:22.804879 26440 webserver.cc:460] Webserver started at http://127.3.147.1:40061/ using document root <none> and password file <none>
I20260812 09:44:22.805477 26440 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:22.805545 26440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:22.805750 26440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:22.807825 26440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/data/instance:
uuid: "93f8a5bf170140e4aae7661852d4dd10"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "fbca307d604604fa33031af1f3d6b35e"
server_key_iv: "f2ac3155fc30d65875138d7625bd98c4"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.808462 26440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance:
uuid: "93f8a5bf170140e4aae7661852d4dd10"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "fbca307d604604fa33031af1f3d6b35e"
server_key_iv: "f2ac3155fc30d65875138d7625bd98c4"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.812348 26440 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 09:44:22.814875 26456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:22.816105 26440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:22.816303 26440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/wal
uuid: "93f8a5bf170140e4aae7661852d4dd10"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "fbca307d604604fa33031af1f3d6b35e"
server_key_iv: "f2ac3155fc30d65875138d7625bd98c4"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.816417 26440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:22.825456 26440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:22.826016 26440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:22.826157 26440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:22.826659 26440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:22.827498 26440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:22.827544 26440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:22.827581 26440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:22.827605 26440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:22.836475 26440 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.1:38809
I20260812 09:44:22.836506 26590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.1:38809 every 8 connection(s)
I20260812 09:44:22.837366 26440 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/data/info.pb
I20260812 09:44:22.845867 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 26440
I20260812 09:44:22.846005 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-0/wal/instance
I20260812 09:44:22.846380 3660 external_mini_cluster.cc:1428] Setting key d1e01a574a6c2ed0192930dbd9fc9974
I20260812 09:44:22.847363 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.839293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:56461 (local address 127.3.147.62:40431)
0812 09:44:22.839438 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:22.839442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:22.840522 (+ 1080us) server_negotiation.cc:407] Connection header received
0812 09:44:22.841921 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:22.841925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:22.841982 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:22.842091 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:22.842930 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.843505 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.844625 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.844871 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.844974 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.846094 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:22.846100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:22.846102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:22.846117 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:22.846150 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:22.846163 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:22.846254 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:22.847176 (+ 922us) server_negotiation.cc:299] Negotiation successful
0812 09:44:22.847232 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 09:44:22.848091 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:22.839670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40431 (local address 127.3.147.1:56461)
0812 09:44:22.840352 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:22.840409 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:22.841681 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:22.842265 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:22.842272 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:22.842349 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:22.842787 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.842798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.843635 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.843640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.844493 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:22.844502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:22.845024 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:22.845028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:22.845181 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:22.845815 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:22.845816 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:22.845852 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:22.845909 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:22.846981 (+ 1072us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:22.846991 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:22.846997 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 09:44:22.847126 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 09:44:22.847342 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":119,"threads_started":1}
I20260812 09:44:22.848400 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.2:0
--local_ip_for_outbound_sockets=127.3.147.2
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40431
--builtin_ntp_servers=127.3.147.20:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 09:44:22.849308 26591 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40431
I20260812 09:44:22.849537 26591 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:22.850150 26591 heartbeater.cc:511] Master 127.3.147.62:40431 requested a full tablet report, sending...
I20260812 09:44:22.851729 26370 ts_manager.cc:194] Registered new tserver with Master: 93f8a5bf170140e4aae7661852d4dd10 (127.3.147.1:38809)
I20260812 09:44:22.853413 26370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.1:56461
W20260812 09:44:22.960379 26594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:22.960638 26594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:22.960685 26594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:22.964581 26594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:22.964620 26594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:22.964668 26594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.2
I20260812 09:44:22.969133 26594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.147.2
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.2
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:22.970026 26594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:22.971251 26594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:22.980409 26603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:22.980435 26601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:22.980409 26600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:22.980753 26594 server_base.cc:1034] running on GCE node
I20260812 09:44:22.981166 26594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:22.981784 26594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:22.982983 26594 hybrid_clock.cc:648] HybridClock initialized: now 1786527862982968 us; error 25 us; skew 500 ppm
I20260812 09:44:22.984830 26594 webserver.cc:460] Webserver started at http://127.3.147.2:43809/ using document root <none> and password file <none>
I20260812 09:44:22.985414 26594 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:22.985504 26594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:22.985719 26594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:22.987700 26594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/data/instance:
uuid: "95601ef2f6bd48a3a7e91b395856f2b1"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "1c68a45b5e75d9bda24854f3b3174151"
server_key_iv: "abcb30e9a98feddc271bba8b1aafcf5a"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.988314 26594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance:
uuid: "95601ef2f6bd48a3a7e91b395856f2b1"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "1c68a45b5e75d9bda24854f3b3174151"
server_key_iv: "abcb30e9a98feddc271bba8b1aafcf5a"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.991894 26594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:44:22.994329 26610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:22.995554 26594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 09:44:22.995687 26594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/wal
uuid: "95601ef2f6bd48a3a7e91b395856f2b1"
format_stamp: "Formatted at 2026-08-12 09:44:22 on dist-test-slave-cbbz"
server_key: "1c68a45b5e75d9bda24854f3b3174151"
server_key_iv: "abcb30e9a98feddc271bba8b1aafcf5a"
server_key_version: "encryptionkey@0"
I20260812 09:44:22.995800 26594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:23.008941 26594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:23.009562 26594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:23.009758 26594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:23.010335 26594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:23.011250 26594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:23.011319 26594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:23.011389 26594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:23.011439 26594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:23.020614 26594 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.2:36503
I20260812 09:44:23.020651 26740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.2:36503 every 8 connection(s)
I20260812 09:44:23.021476 26594 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/data/info.pb
I20260812 09:44:23.025519 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 26594
I20260812 09:44:23.025687 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/wal/instance
I20260812 09:44:23.025988 3660 external_mini_cluster.cc:1428] Setting key 36428e71745ff39788627ed9993d6b7b
I20260812 09:44:23.028100 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.147.3:0
--local_ip_for_outbound_sockets=127.3.147.3
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40431
--builtin_ntp_servers=127.3.147.20:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 09:44:23.031508 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.023312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.2:34989 (local address 127.3.147.62:40431)
0812 09:44:23.023456 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.023460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.024511 (+ 1051us) server_negotiation.cc:407] Connection header received
0812 09:44:23.026006 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.026010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.026073 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.026180 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:23.027164 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.027844 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.029207 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.029600 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.029720 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.031016 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:23.031020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:23.031021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:23.031033 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:23.031069 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:23.031077 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:23.031144 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.031350 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.031401 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 09:44:23.032709 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.023690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40431 (local address 127.3.147.2:34989)
0812 09:44:23.024330 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:23.024368 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:23.025786 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:23.026368 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:23.026375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:23.026457 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:23.027018 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.027031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.027990 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.027992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.029071 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.029082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.029781 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.029787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.030032 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.030702 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:23.030705 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:23.030760 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:23.030857 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:23.031175 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:23.031187 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:23.031195 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 09:44:23.031340 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 09:44:23.031619 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":112,"threads_started":1}
I20260812 09:44:23.033944 26741 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40431
I20260812 09:44:23.034176 26741 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:23.034672 26741 heartbeater.cc:511] Master 127.3.147.62:40431 requested a full tablet report, sending...
I20260812 09:44:23.035674 26370 ts_manager.cc:194] Registered new tserver with Master: 95601ef2f6bd48a3a7e91b395856f2b1 (127.3.147.2:36503)
I20260812 09:44:23.036307 26370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.2:34989
W20260812 09:44:23.140612 26745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:23.140832 26745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:23.140867 26745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:23.144943 26745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:23.144989 26745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:23.145069 26745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.147.3
I20260812 09:44:23.149705 26745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.147.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.147.3
--webserver_port=0
--tserver_master_addrs=127.3.147.62:40431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.147.3
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:23.150740 26745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:23.152186 26745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:23.161791 26754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:23.161787 26752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:23.161821 26751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:23.161837 26745 server_base.cc:1034] running on GCE node
I20260812 09:44:23.162428 26745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:23.163022 26745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:23.164206 26745 hybrid_clock.cc:648] HybridClock initialized: now 1786527863164180 us; error 38 us; skew 500 ppm
I20260812 09:44:23.165966 26745 webserver.cc:460] Webserver started at http://127.3.147.3:34867/ using document root <none> and password file <none>
I20260812 09:44:23.166502 26745 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:23.166563 26745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:23.166817 26745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:23.168824 26745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/data/instance:
uuid: "6a9b10a2974648bbbccbb6e1400f66de"
format_stamp: "Formatted at 2026-08-12 09:44:23 on dist-test-slave-cbbz"
server_key: "3b7d9fdbad2d10f706fe6352f04dee14"
server_key_iv: "a83217f3aec52c848a510d89230d0e04"
server_key_version: "encryptionkey@0"
I20260812 09:44:23.169361 26745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance:
uuid: "6a9b10a2974648bbbccbb6e1400f66de"
format_stamp: "Formatted at 2026-08-12 09:44:23 on dist-test-slave-cbbz"
server_key: "3b7d9fdbad2d10f706fe6352f04dee14"
server_key_iv: "a83217f3aec52c848a510d89230d0e04"
server_key_version: "encryptionkey@0"
I20260812 09:44:23.173008 26745 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:23.175267 26762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:23.176486 26745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:23.176625 26745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/wal
uuid: "6a9b10a2974648bbbccbb6e1400f66de"
format_stamp: "Formatted at 2026-08-12 09:44:23 on dist-test-slave-cbbz"
server_key: "3b7d9fdbad2d10f706fe6352f04dee14"
server_key_iv: "a83217f3aec52c848a510d89230d0e04"
server_key_version: "encryptionkey@0"
I20260812 09:44:23.176776 26745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:23.202584 26745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:23.203356 26745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:23.203500 26745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:23.204095 26745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 09:44:23.205060 26745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 09:44:23.205108 26745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:23.205147 26745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 09:44:23.205163 26745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:23.214438 26745 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.3:38435
I20260812 09:44:23.214473 26895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.3:38435 every 8 connection(s)
I20260812 09:44:23.215293 26745 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/data/info.pb
I20260812 09:44:23.216325 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 26745
I20260812 09:44:23.216445 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/wal/instance
I20260812 09:44:23.216789 3660 external_mini_cluster.cc:1428] Setting key 1157b5f187073add2cd44978da67c43e
I20260812 09:44:23.224711 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.217297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.3:46725 (local address 127.3.147.62:40431)
0812 09:44:23.217462 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.217467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.218718 (+ 1251us) server_negotiation.cc:407] Connection header received
0812 09:44:23.220332 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.220336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.220393 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.220473 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:23.221327 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.221858 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.222929 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.223136 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.223208 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.224320 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:23.224323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:23.224324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:23.224334 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:23.224350 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:23.224356 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:23.224413 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.224558 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.224604 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 09:44:23.225525 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.217746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:40431 (local address 127.3.147.3:46725)
0812 09:44:23.218562 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:23.218619 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:23.220081 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:23.220646 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:23.220652 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:23.220732 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:23.221208 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.221221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.221971 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.221973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.222828 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.222837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.223270 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.223273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.223406 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.224043 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 09:44:23.224045 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:23.224082 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 09:44:23.224186 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:23.224429 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:23.224436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:23.224441 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 09:44:23.224543 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 09:44:23.224775 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":154,"threads_started":1}
I20260812 09:44:23.226606 26896 heartbeater.cc:347] Connected to a master server at 127.3.147.62:40431
I20260812 09:44:23.226838 26896 heartbeater.cc:464] Registering TS with master...
I20260812 09:44:23.227365 26896 heartbeater.cc:511] Master 127.3.147.62:40431 requested a full tablet report, sending...
I20260812 09:44:23.228401 26370 ts_manager.cc:194] Registered new tserver with Master: 6a9b10a2974648bbbccbb6e1400f66de (127.3.147.3:38435)
I20260812 09:44:23.228909 26370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.147.3:46725
I20260812 09:44:23.231015 3660 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 09:44:23.237176 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.233122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.3.147.62:40431)
0812 09:44:23.233255 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.233262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.233478 (+ 216us) server_negotiation.cc:407] Connection header received
0812 09:44:23.233643 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.233646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.233693 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.233805 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:23.234104 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.234605 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.235499 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.235739 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.235874 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.236690 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:23.236694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:23.236697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:23.236711 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:23.236735 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:23.236750 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:23.236838 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.236969 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.237026 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:44:23.239885 26370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 09:44:23.242883 26370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 09:44:23.257354 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.251394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:38809 (local address 127.0.0.1:40290)
0812 09:44:23.252098 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:23.252112 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:23.252228 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:23.252834 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:23.252837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:23.252853 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:23.252970 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.252979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.254342 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.254345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.255706 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.255717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.256868 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.256872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.256961 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.256983 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 09:44:23.257037 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 09:44:23.257187 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":86,"thread_start_us":116,"threads_started":1}
I20260812 09:44:23.257367 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.252410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.3.147.2:36503)
0812 09:44:23.252718 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.252731 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.253202 (+ 471us) server_negotiation.cc:407] Connection header received
0812 09:44:23.253291 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.253296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.253477 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.253620 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:23.253873 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.254752 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.255999 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.256634 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.256828 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.257019 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.257120 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.257210 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":111,"threads_started":1}
I20260812 09:44:23.257396 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.251209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:38435 (local address 127.0.0.1:39004)
0812 09:44:23.251694 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:23.251739 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:23.251881 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:23.252693 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:23.252699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:23.252728 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:23.252873 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.252881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.254199 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.254205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.255709 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.255717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.257052 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.257055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.257137 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.257158 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 09:44:23.257226 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 09:44:23.257285 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":44,"thread_start_us":80,"threads_started":1}
I20260812 09:44:23.257411 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.251898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:36503 (local address 127.0.0.1:39740)
0812 09:44:23.253076 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:23.253088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:23.253180 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:23.253637 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:23.253652 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:23.253671 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:23.253754 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.253759 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.254871 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.254877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.255867 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.255877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.256779 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.256783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.256899 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.256935 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 09:44:23.257036 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 09:44:23.257190 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"mutex_wait_us":34,"thread_start_us":95,"threads_started":1}
I20260812 09:44:23.258131 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.251363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.3.147.3:38435)
0812 09:44:23.252180 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.252194 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.252222 (+ 28us) server_negotiation.cc:407] Connection header received
0812 09:44:23.252304 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.252309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.252508 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.253031 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:23.253190 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.254082 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.255857 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.256684 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.256793 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.256991 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.257868 (+ 877us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.257976 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":127,"threads_started":1}
I20260812 09:44:23.258239 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.251937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.3.147.1:38809)
0812 09:44:23.252367 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.252377 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.252397 (+ 20us) server_negotiation.cc:407] Connection header received
0812 09:44:23.252460 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.252464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.252658 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.252843 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:23.253107 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.254219 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.255858 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.256681 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.257575 (+ 894us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.257776 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.257836 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.257964 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":111,"threads_started":1}
I20260812 09:44:23.260586 26664 tablet_service.cc:1467] Processing CreateTablet for tablet 26670506a2844dc4bbb0ff262479db3c (DEFAULT_TABLE table=test-table [id=45ac030a2e5342a398381f1197d06008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:23.260871 26511 tablet_service.cc:1467] Processing CreateTablet for tablet 26670506a2844dc4bbb0ff262479db3c (DEFAULT_TABLE table=test-table [id=45ac030a2e5342a398381f1197d06008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:23.260857 26822 tablet_service.cc:1467] Processing CreateTablet for tablet 26670506a2844dc4bbb0ff262479db3c (DEFAULT_TABLE table=test-table [id=45ac030a2e5342a398381f1197d06008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 09:44:23.261603 26664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26670506a2844dc4bbb0ff262479db3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:23.261876 26511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26670506a2844dc4bbb0ff262479db3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:23.261876 26822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26670506a2844dc4bbb0ff262479db3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:23.269376 26918 tablet_bootstrap.cc:492] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: Bootstrap starting.
I20260812 09:44:23.270180 26920 tablet_bootstrap.cc:492] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: Bootstrap starting.
I20260812 09:44:23.270608 26919 tablet_bootstrap.cc:492] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10: Bootstrap starting.
I20260812 09:44:23.271639 26918 tablet_bootstrap.cc:654] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:23.272318 26920 tablet_bootstrap.cc:654] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:23.272529 26918 log.cc:824] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:23.272619 26919 tablet_bootstrap.cc:654] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:23.273140 26920 log.cc:824] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:23.273350 26919 log.cc:824] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:23.275226 26920 tablet_bootstrap.cc:492] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: No bootstrap required, opened a new log
I20260812 09:44:23.275419 26920 ts_tablet_manager.cc:1397] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 09:44:23.275529 26919 tablet_bootstrap.cc:492] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10: No bootstrap required, opened a new log
I20260812 09:44:23.275712 26919 ts_tablet_manager.cc:1397] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 09:44:23.277326 26918 tablet_bootstrap.cc:492] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: No bootstrap required, opened a new log
I20260812 09:44:23.277514 26918 ts_tablet_manager.cc:1397] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 09:44:23.278759 26920 raft_consensus.cc:348] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.278990 26920 raft_consensus.cc:374] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:23.279050 26920 raft_consensus.cc:729] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9b10a2974648bbbccbb6e1400f66de, State: Initialized, Role: FOLLOWER
I20260812 09:44:23.279526 26920 consensus_queue.cc:260] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.279716 26919 raft_consensus.cc:348] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.279966 26919 raft_consensus.cc:374] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:23.280015 26919 raft_consensus.cc:729] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f8a5bf170140e4aae7661852d4dd10, State: Initialized, Role: FOLLOWER
I20260812 09:44:23.280432 26919 consensus_queue.cc:260] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.280577 26920 ts_tablet_manager.cc:1428] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 09:44:23.280849 26896 heartbeater.cc:503] Master 127.3.147.62:40431 was elected leader, sending a full tablet report...
I20260812 09:44:23.281225 26591 heartbeater.cc:503] Master 127.3.147.62:40431 was elected leader, sending a full tablet report...
I20260812 09:44:23.281432 26919 ts_tablet_manager.cc:1428] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 09:44:23.281661 26918 raft_consensus.cc:348] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.281917 26918 raft_consensus.cc:374] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:23.281973 26918 raft_consensus.cc:729] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95601ef2f6bd48a3a7e91b395856f2b1, State: Initialized, Role: FOLLOWER
I20260812 09:44:23.282379 26918 consensus_queue.cc:260] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.283025 26741 heartbeater.cc:503] Master 127.3.147.62:40431 was elected leader, sending a full tablet report...
I20260812 09:44:23.283301 26918 ts_tablet_manager.cc:1428] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260812 09:44:23.338620 26592 tablet.cc:2369] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:23.370025 26926 raft_consensus.cc:482] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 09:44:23.370270 26926 raft_consensus.cc:504] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.371398 26926 leader_election.cc:288] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a9b10a2974648bbbccbb6e1400f66de (127.3.147.3:38435), 95601ef2f6bd48a3a7e91b395856f2b1 (127.3.147.2:36503)
I20260812 09:44:23.376076 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.371746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.3:38435 (local address 127.3.147.1:34125)
0812 09:44:23.372011 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:23.372028 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:23.372158 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:23.372462 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:23.372466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:23.372489 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:23.372624 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.372633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.373624 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.373627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.374861 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.374872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.375670 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.375673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.375736 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.375756 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 09:44:23.375818 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 09:44:23.375911 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":5,"thread_start_us":99,"threads_started":1}
I20260812 09:44:23.376067 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.371865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:34125 (local address 127.3.147.3:38435)
0812 09:44:23.372005 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.372009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.372104 (+ 95us) server_negotiation.cc:407] Connection header received
0812 09:44:23.372270 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.372273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.372330 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.372416 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:23.372747 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.373515 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.374982 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.375556 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.375671 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.375734 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.375860 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.375933 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 09:44:23.376384 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.372114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.2:36503 (local address 127.3.147.1:37951)
0812 09:44:23.372571 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:23.372583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:23.372674 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:23.372918 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:23.372920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:23.372934 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 09:44:23.373033 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.373037 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.373905 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.373911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.375075 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:23.375082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.375937 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.375941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:23.376030 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.376050 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 09:44:23.376130 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 09:44:23.376216 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":36,"thread_start_us":60,"threads_started":1}
I20260812 09:44:23.376657 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.372287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.147.1:37951 (local address 127.3.147.2:36503)
0812 09:44:23.372454 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.372458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.372662 (+ 204us) server_negotiation.cc:407] Connection header received
0812 09:44:23.372761 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.372764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.372817 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.372898 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 09:44:23.373164 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.373796 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.375196 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.375764 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.375895 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.375949 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.376462 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.376544 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 09:44:23.376744 26843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26670506a2844dc4bbb0ff262479db3c" candidate_uuid: "93f8a5bf170140e4aae7661852d4dd10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a9b10a2974648bbbccbb6e1400f66de" is_pre_election: true
I20260812 09:44:23.377072 26843 raft_consensus.cc:2454] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f8a5bf170140e4aae7661852d4dd10 in term 0.
I20260812 09:44:23.377290 26687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26670506a2844dc4bbb0ff262479db3c" candidate_uuid: "93f8a5bf170140e4aae7661852d4dd10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" is_pre_election: true
I20260812 09:44:23.377588 26687 raft_consensus.cc:2454] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f8a5bf170140e4aae7661852d4dd10 in term 0.
I20260812 09:44:23.377581 26460 leader_election.cc:302] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9b10a2974648bbbccbb6e1400f66de, 93f8a5bf170140e4aae7661852d4dd10; no voters:
I20260812 09:44:23.377889 26926 raft_consensus.cc:2781] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 09:44:23.378005 26926 raft_consensus.cc:482] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 09:44:23.378065 26926 raft_consensus.cc:3037] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:23.379197 26926 raft_consensus.cc:504] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.379596 26926 leader_election.cc:288] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [CANDIDATE]: Term 1 election: Requested vote from peers 6a9b10a2974648bbbccbb6e1400f66de (127.3.147.3:38435), 95601ef2f6bd48a3a7e91b395856f2b1 (127.3.147.2:36503)
I20260812 09:44:23.379915 26843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26670506a2844dc4bbb0ff262479db3c" candidate_uuid: "93f8a5bf170140e4aae7661852d4dd10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a9b10a2974648bbbccbb6e1400f66de"
I20260812 09:44:23.379972 26687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26670506a2844dc4bbb0ff262479db3c" candidate_uuid: "93f8a5bf170140e4aae7661852d4dd10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95601ef2f6bd48a3a7e91b395856f2b1"
I20260812 09:44:23.380071 26687 raft_consensus.cc:3037] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:23.380059 26843 raft_consensus.cc:3037] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:23.381197 26843 raft_consensus.cc:2454] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f8a5bf170140e4aae7661852d4dd10 in term 1.
I20260812 09:44:23.381315 26687 raft_consensus.cc:2454] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f8a5bf170140e4aae7661852d4dd10 in term 1.
I20260812 09:44:23.381543 26460 leader_election.cc:302] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9b10a2974648bbbccbb6e1400f66de, 93f8a5bf170140e4aae7661852d4dd10; no voters:
I20260812 09:44:23.381747 26926 raft_consensus.cc:2781] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:23.382011 26926 raft_consensus.cc:686] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 1 LEADER]: Becoming Leader. State: Replica: 93f8a5bf170140e4aae7661852d4dd10, State: Running, Role: LEADER
I20260812 09:44:23.382369 26926 consensus_queue.cc:237] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } }
I20260812 09:44:23.385478 26368 catalog_manager.cc:5495] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93f8a5bf170140e4aae7661852d4dd10 (127.3.147.1). New cstate: current_term: 1 leader_uuid: "93f8a5bf170140e4aae7661852d4dd10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93f8a5bf170140e4aae7661852d4dd10" member_type: VOTER last_known_addr { host: "127.3.147.1" port: 38809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 } health_report { overall_health: UNKNOWN } } }
I20260812 09:44:23.407630 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.404469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40298 (local address 127.3.147.1:38809)
0812 09:44:23.404587 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:23.404591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:23.404605 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:44:23.404722 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:23.404724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:23.404776 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:23.404864 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:23.405078 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.405578 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.406377 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:23.406581 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:23.406659 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:23.406904 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:23.407294 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:23.407401 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:23.407486 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 09:44:23.407529 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 09:44:23.413251 26687 raft_consensus.cc:1261] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 1 FOLLOWER]: Refusing update from remote peer 93f8a5bf170140e4aae7661852d4dd10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:23.413271 26843 raft_consensus.cc:1261] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 1 FOLLOWER]: Refusing update from remote peer 93f8a5bf170140e4aae7661852d4dd10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:23.414028 26926 consensus_queue.cc:1035] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95601ef2f6bd48a3a7e91b395856f2b1" member_type: VOTER last_known_addr { host: "127.3.147.2" port: 36503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:23.414278 26932 consensus_queue.cc:1035] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a9b10a2974648bbbccbb6e1400f66de" member_type: VOTER last_known_addr { host: "127.3.147.3" port: 38435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:23.423897 26938 mvcc.cc:205] Tried to move back new op lower bound from 7317618128533512192 to 7317618128416620544. Current Snapshot: MvccSnapshot[applied={T|T < 7317618128533512192}]
I20260812 09:44:23.428663 26937 mvcc.cc:205] Tried to move back new op lower bound from 7317618128533512192 to 7317618128416620544. Current Snapshot: MvccSnapshot[applied={T|T < 7317618128533512192 or (T in {7317618128533512192})}]
I20260812 09:44:23.430707 26370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:23.430878 26370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 09:44:23.433420 26370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 671d0cc603b5482d98682a79d932defd: Sending DeleteTablet for 3 replicas of tablet 26670506a2844dc4bbb0ff262479db3c
I20260812 09:44:23.434191 26822 tablet_service.cc:1514] Processing DeleteTablet for tablet 26670506a2844dc4bbb0ff262479db3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:23 UTC) from {username='slave'} at 127.0.0.1:39004
I20260812 09:44:23.434281 26664 tablet_service.cc:1514] Processing DeleteTablet for tablet 26670506a2844dc4bbb0ff262479db3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:23 UTC) from {username='slave'} at 127.0.0.1:39740
I20260812 09:44:23.434307 26511 tablet_service.cc:1514] Processing DeleteTablet for tablet 26670506a2844dc4bbb0ff262479db3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 09:44:23 UTC) from {username='slave'} at 127.0.0.1:40290
I20260812 09:44:23.434711 26946 tablet_replica.cc:330] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10: stopping tablet replica
I20260812 09:44:23.434758 26947 tablet_replica.cc:330] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: stopping tablet replica
I20260812 09:44:23.435078 26946 raft_consensus.cc:2229] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 1 LEADER]: Raft consensus shutting down.
I20260812 09:44:23.435081 26947 raft_consensus.cc:2229] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:23.435412 26947 raft_consensus.cc:2258] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:23.435431 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 26440
I20260812 09:44:23.435623 26946 raft_consensus.cc:2258] T 26670506a2844dc4bbb0ff262479db3c P 93f8a5bf170140e4aae7661852d4dd10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:23.436158 26945 tablet_replica.cc:330] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: stopping tablet replica
I20260812 09:44:23.436517 26945 raft_consensus.cc:2229] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 09:44:23.436815 26945 raft_consensus.cc:2258] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 09:44:23.437156 26947 ts_tablet_manager.cc:1905] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:23.438459 26945 ts_tablet_manager.cc:1905] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 09:44:23.440634 26947 ts_tablet_manager.cc:1918] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:23.440744 26947 log.cc:1196] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-1/wal/wals/26670506a2844dc4bbb0ff262479db3c
I20260812 09:44:23.440903 26945 ts_tablet_manager.cc:1918] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 09:44:23.441013 26945 log.cc:1196] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: Deleting WAL directory at /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786527603645036-3660-0/minicluster-data/ts-2/wal/wals/26670506a2844dc4bbb0ff262479db3c
I20260812 09:44:23.441082 26947 ts_tablet_manager.cc:1939] T 26670506a2844dc4bbb0ff262479db3c P 95601ef2f6bd48a3a7e91b395856f2b1: Deleting consensus metadata
I20260812 09:44:23.441329 26945 ts_tablet_manager.cc:1939] T 26670506a2844dc4bbb0ff262479db3c P 6a9b10a2974648bbbccbb6e1400f66de: Deleting consensus metadata
I20260812 09:44:23.442442 26356 catalog_manager.cc:4841] TS 95601ef2f6bd48a3a7e91b395856f2b1 (127.3.147.2:36503): tablet 26670506a2844dc4bbb0ff262479db3c (table test-table [id=45ac030a2e5342a398381f1197d06008]) successfully deleted
I20260812 09:44:23.442639 26356 catalog_manager.cc:4841] TS 6a9b10a2974648bbbccbb6e1400f66de (127.3.147.3:38435): tablet 26670506a2844dc4bbb0ff262479db3c (table test-table [id=45ac030a2e5342a398381f1197d06008]) successfully deleted
W20260812 09:44:23.445554 26355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.147.1:38809 (error 108)
I20260812 09:44:23.445737 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 26594
I20260812 09:44:23.451737 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:23.451340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.1:38809 (local address 127.0.0.1:40304)
0812 09:44:23.451510 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:23.451601 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.1:38809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 09:44:23.451920 26355 catalog_manager.cc:4584] TS 93f8a5bf170140e4aae7661852d4dd10 (127.3.147.1:38809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26670506a2844dc4bbb0ff262479db3c: Network error: Client connection negotiation failed: client connection to 127.3.147.1:38809: connect: Connection refused (error 111)
I20260812 09:44:23.455965 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 26745
I20260812 09:44:23.463469 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 26332
2026-08-12T09:44:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1086 ms)
[----------] 6 tests from EncryptionPolicyTest (10791 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:23 dist-test-slave-cbbz krb5kdc[26951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:23 dist-test-slave-cbbz krb5kdc[26951](info): set up 2 sockets
Aug 12 09:44:23 dist-test-slave-cbbz krb5kdc[26951](info): commencing operation
krb5kdc: starting...
W20260812 09:44:24.567301 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:24 dist-test-slave-cbbz krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:24.694938 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:24.804255 26967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:24.804502 26967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:24.804543 26967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:24.808403 26967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:24.808449 26967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:24.808467 26967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:24.808483 26967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:24.808498 26967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:24.813529 26967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43297
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:24.814481 26967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:24.815402 26967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:24.824386 26973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:24.824380 26975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:24.824378 26972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:24.824429 26967 server_base.cc:1034] running on GCE node
I20260812 09:44:24.825351 26967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:24.826324 26967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:24.827483 26967 hybrid_clock.cc:648] HybridClock initialized: now 1786527864827460 us; error 42 us; skew 500 ppm
Aug 12 09:44:24 dist-test-slave-cbbz krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:24.830202 26967 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:24.830994 26967 webserver.cc:460] Webserver started at http://127.0.0.1:45157/ using document root <none> and password file <none>
I20260812 09:44:24.831559 26967 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:24.831661 26967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:24.831900 26967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:24.833936 26967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "476865bf2a034579b8880416b67f70ed"
format_stamp: "Formatted at 2026-08-12 09:44:24 on dist-test-slave-cbbz"
server_key: "2642c92adf7471958dd9956d592b839b"
server_key_iv: "3dc3ef0e829eecc70ab7a36440b181b8"
server_key_version: "encryptionkey@0"
I20260812 09:44:24.834494 26967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "476865bf2a034579b8880416b67f70ed"
format_stamp: "Formatted at 2026-08-12 09:44:24 on dist-test-slave-cbbz"
server_key: "2642c92adf7471958dd9956d592b839b"
server_key_iv: "3dc3ef0e829eecc70ab7a36440b181b8"
server_key_version: "encryptionkey@0"
I20260812 09:44:24.838090 26967 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:44:24.840378 26984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:24.841548 26967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:24.841696 26967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "476865bf2a034579b8880416b67f70ed"
format_stamp: "Formatted at 2026-08-12 09:44:24 on dist-test-slave-cbbz"
server_key: "2642c92adf7471958dd9956d592b839b"
server_key_iv: "3dc3ef0e829eecc70ab7a36440b181b8"
server_key_version: "encryptionkey@0"
I20260812 09:44:24.841818 26967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:24.855195 26967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:24.857992 26967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:24.858234 26967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:24.865530 26967 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43297
I20260812 09:44:24.865557 27044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43297 every 8 connection(s)
I20260812 09:44:24.866454 26967 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:24.869131 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:24.870757 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 26967
I20260812 09:44:24.870908 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:24.871232 3660 external_mini_cluster.cc:1428] Setting key 0c68e300f55e5bbfa7f3bf477301a9b1
I20260812 09:44:24.875273 27045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed: Bootstrap starting.
I20260812 09:44:24.877526 27045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:24.878314 27045 log.cc:824] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:44:24 dist-test-slave-cbbz krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 09:44:24.880529 27045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed: No bootstrap required, opened a new log
I20260812 09:44:24.884867 27045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476865bf2a034579b8880416b67f70ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } }
I20260812 09:44:24.885074 27045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:24.885141 27045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 476865bf2a034579b8880416b67f70ed, State: Initialized, Role: FOLLOWER
I20260812 09:44:24.885489 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:24.872673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34255 (local address 127.0.0.1:43297)
0812 09:44:24.873085 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:24.873100 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:24.873134 (+ 34us) server_negotiation.cc:407] Connection header received
0812 09:44:24.873767 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:24.873782 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:24.874096 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:24.874424 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:24.875277 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:24.876349 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:24.877192 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:24.877698 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:24.877870 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:24.880749 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:24.880754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:24.880767 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:24.880793 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:24.882424 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:24.883017 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:24.883021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:24.883026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:24.883140 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:24.883503 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:24.883506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:24.883507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:24.883874 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:24.884051 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:24.884400 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 09:44:24.884661 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":126,"threads_started":1}
I20260812 09:44:24.885623 27045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476865bf2a034579b8880416b67f70ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } }
I20260812 09:44:24.885774 27045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:24.885829 27045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:24.885900 27045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:24.886847 27045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476865bf2a034579b8880416b67f70ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } }
I20260812 09:44:24.887202 27045 leader_election.cc:302] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 476865bf2a034579b8880416b67f70ed; no voters:
I20260812 09:44:24.887607 27045 leader_election.cc:288] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:24.887734 27050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:24.887964 27050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [term 1 LEADER]: Becoming Leader. State: Replica: 476865bf2a034579b8880416b67f70ed, State: Running, Role: LEADER
I20260812 09:44:24.888309 27050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476865bf2a034579b8880416b67f70ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } }
I20260812 09:44:24.888666 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:24.888922 27045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:24.890022 27052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 476865bf2a034579b8880416b67f70ed. Latest consensus state: current_term: 1 leader_uuid: "476865bf2a034579b8880416b67f70ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476865bf2a034579b8880416b67f70ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } } }
I20260812 09:44:24.890167 27052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:24.890519 27051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "476865bf2a034579b8880416b67f70ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476865bf2a034579b8880416b67f70ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } } }
I20260812 09:44:24.890610 27051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed [sys.catalog]: This master's current role is: LEADER
W20260812 09:44:24.894059 27073 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:24.894153 27073 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:24.894208 27064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:24.894956 27064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:24.900419 27064 catalog_manager.cc:1347] Generated new cluster ID: b7c2b30c3a0c490db7d54da0f806b5d5
I20260812 09:44:24.900487 27064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:24.901058 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:24.891013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60117 (local address 127.0.0.1:43297)
0812 09:44:24.891204 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:24.891211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:24.891655 (+ 444us) server_negotiation.cc:407] Connection header received
0812 09:44:24.891798 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:24.891801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:24.891848 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:24.891930 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:24.892803 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:24.893537 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:24.894603 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:24.894983 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:24.895089 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:24.896994 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:24.896999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:24.897002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:24.897030 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:24.898830 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:24.899402 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:24.899405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:24.899407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:24.899496 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:24.899889 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:24.899892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:24.899894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:24.900104 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:24.900283 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:24.900803 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 09:44:24.900900 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260812 09:44:24.908838 27064 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:24.909962 27064 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:24.921681 27064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 476865bf2a034579b8880416b67f70ed: Generated new TSK 0
I20260812 09:44:24.922329 27064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:24.927594 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 26967
W20260812 09:44:24.936259 26949 connection.cc:535] client connection to 127.0.0.1:43297 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43297): Connection reset by peer (error 104)
2026-08-12T09:44:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:24 dist-test-slave-cbbz krb5kdc[27081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:24 dist-test-slave-cbbz krb5kdc[27081](info): set up 2 sockets
Aug 12 09:44:24 dist-test-slave-cbbz krb5kdc[27081](info): commencing operation
krb5kdc: starting...
W20260812 09:44:26.044332 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:26 dist-test-slave-cbbz krb5kdc[27081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:26.171147 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:26.286597 27097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:26.286857 27097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:26.286902 27097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:26.291007 27097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:26.291065 27097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:26.291085 27097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:26.291101 27097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:26.291146 27097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:26.296617 27097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:26.297657 27097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:26.298681 27097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:26.307384 27102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:26.307363 27105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:26.307356 27103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:26.307823 27097 server_base.cc:1034] running on GCE node
I20260812 09:44:26.308465 27097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:26.309527 27097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:26.310710 27097 hybrid_clock.cc:648] HybridClock initialized: now 1786527866310691 us; error 30 us; skew 500 ppm
Aug 12 09:44:26 dist-test-slave-cbbz krb5kdc[27081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:26.313525 27097 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:26.314424 27097 webserver.cc:460] Webserver started at http://127.0.0.1:43087/ using document root <none> and password file <none>
I20260812 09:44:26.314954 27097 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:26.315042 27097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:26.315274 27097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:26.317391 27097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "23f8f654445846a98a2266968df09c94"
format_stamp: "Formatted at 2026-08-12 09:44:26 on dist-test-slave-cbbz"
server_key: "7a1784df93d4226c742da71c53de62f8"
server_key_iv: "938ed17fab0b958c5333f76cd5ddf62e"
server_key_version: "encryptionkey@0"
I20260812 09:44:26.317946 27097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "23f8f654445846a98a2266968df09c94"
format_stamp: "Formatted at 2026-08-12 09:44:26 on dist-test-slave-cbbz"
server_key: "7a1784df93d4226c742da71c53de62f8"
server_key_iv: "938ed17fab0b958c5333f76cd5ddf62e"
server_key_version: "encryptionkey@0"
I20260812 09:44:26.321642 27097 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:26.323904 27113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:26.325095 27097 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:26.325244 27097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "23f8f654445846a98a2266968df09c94"
format_stamp: "Formatted at 2026-08-12 09:44:26 on dist-test-slave-cbbz"
server_key: "7a1784df93d4226c742da71c53de62f8"
server_key_iv: "938ed17fab0b958c5333f76cd5ddf62e"
server_key_version: "encryptionkey@0"
I20260812 09:44:26.325363 27097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:26.343673 27097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:26.346858 27097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:26.347081 27097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:26.354432 27097 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42591
I20260812 09:44:26.354449 27176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42591 every 8 connection(s)
I20260812 09:44:26.355415 27097 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:26.357411 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 27097
I20260812 09:44:26.357563 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:26.357911 3660 external_mini_cluster.cc:1428] Setting key 503daef5b9fe08465e078d3679f448d2
I20260812 09:44:26.358359 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:26.364451 27177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94: Bootstrap starting.
I20260812 09:44:26.366828 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:26.359285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54255 (local address 127.0.0.1:42591)
0812 09:44:26.359733 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:26.359760 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:26.359796 (+ 36us) server_negotiation.cc:407] Connection header received
0812 09:44:26.360515 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:26.360532 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:26.360874 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:26.361189 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:26.361596 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:26.362570 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:26.363451 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:26.363911 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:26.364074 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:26.365045 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:26.365049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:26.365061 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:26.365603 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:26.365658 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:26.365681 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:26.365820 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:26.366003 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 09:44:26.366142 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":148,"threads_started":1}
I20260812 09:44:26.366910 27177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:26.367774 27177 log.cc:824] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:26.369732 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:26.369872 27177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94: No bootstrap required, opened a new log
I20260812 09:44:26.373498 27177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23f8f654445846a98a2266968df09c94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42591 } }
I20260812 09:44:26.373703 27177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:26.373764 27177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f8f654445846a98a2266968df09c94, State: Initialized, Role: FOLLOWER
I20260812 09:44:26.374181 27177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23f8f654445846a98a2266968df09c94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42591 } }
I20260812 09:44:26.374341 27177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:26.374437 27177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:26.374572 27177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:26.375560 27177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23f8f654445846a98a2266968df09c94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42591 } }
I20260812 09:44:26.375705 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:26.371642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33999 (local address 127.0.0.1:42591)
0812 09:44:26.371794 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:26.371800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:26.372011 (+ 211us) server_negotiation.cc:407] Connection header received
0812 09:44:26.372131 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:26.372153 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:26.372197 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:26.372280 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:26.372521 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:26.373027 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:26.373929 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:26.374227 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:26.374344 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:26.375256 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:26.375269 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:26.375271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:26.375283 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:26.375312 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:26.375321 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:26.375413 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:26.375531 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 09:44:26.375563 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:26.375957 27177 leader_election.cc:302] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23f8f654445846a98a2266968df09c94; no voters:
I20260812 09:44:26.376312 27177 leader_election.cc:288] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:26.376381 27187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:26.376590 27187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [term 1 LEADER]: Becoming Leader. State: Replica: 23f8f654445846a98a2266968df09c94, State: Running, Role: LEADER
I20260812 09:44:26.376870 27187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23f8f654445846a98a2266968df09c94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42591 } }
I20260812 09:44:26.377539 27177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:26.377990 27187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23f8f654445846a98a2266968df09c94. Latest consensus state: current_term: 1 leader_uuid: "23f8f654445846a98a2266968df09c94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23f8f654445846a98a2266968df09c94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42591 } } }
I20260812 09:44:26.378100 27187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:26.378314 27187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23f8f654445846a98a2266968df09c94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23f8f654445846a98a2266968df09c94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42591 } } }
I20260812 09:44:26.378428 27187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:26.378628 27194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:26.382174 27194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:26.387749 27194 catalog_manager.cc:1347] Generated new cluster ID: 9f8ac7906f344526862c82d6cd3eb5f5
I20260812 09:44:26.387823 27194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:26.397142 27194 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:26.398300 27194 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:26.405823 27194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23f8f654445846a98a2266968df09c94: Generated new TSK 0
I20260812 09:44:26.406622 27194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:26.420598 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 27097
W20260812 09:44:26.429713 27079 connection.cc:535] client connection to 127.0.0.1:42591 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42591): Connection reset by peer (error 104)
2026-08-12T09:44:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:26 dist-test-slave-cbbz krb5kdc[27214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:26 dist-test-slave-cbbz krb5kdc[27214](info): set up 2 sockets
Aug 12 09:44:26 dist-test-slave-cbbz krb5kdc[27214](info): commencing operation
krb5kdc: starting...
W20260812 09:44:27.536391 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:27 dist-test-slave-cbbz krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:27.663415 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:27.776340 27227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:27.776626 27227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:27.776701 27227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:27.780841 27227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:27.780920 27227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:27.780958 27227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:27.781015 27227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:27.781056 27227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:27.786199 27227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:27.787220 27227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:27.788257 27227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:27.796602 27235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:27.796622 27232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:27.796622 27233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:27.797043 27227 server_base.cc:1034] running on GCE node
I20260812 09:44:27.797607 27227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:27.798684 27227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:27.799892 27227 hybrid_clock.cc:648] HybridClock initialized: now 1786527867799875 us; error 30 us; skew 500 ppm
Aug 12 09:44:27 dist-test-slave-cbbz krb5kdc[27214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527867, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:27.802630 27227 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:27.803562 27227 webserver.cc:460] Webserver started at http://127.0.0.1:35647/ using document root <none> and password file <none>
I20260812 09:44:27.804127 27227 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:27.804281 27227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:27.804520 27227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:27.806529 27227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "1adaf2a0e3394868ab8a1047c057b005"
format_stamp: "Formatted at 2026-08-12 09:44:27 on dist-test-slave-cbbz"
server_key: "f2d8c92f5d9c899f99a89146ed2d5a90"
server_key_iv: "58af92446efc51257fc0527e9b0862a3"
server_key_version: "encryptionkey@0"
I20260812 09:44:27.807085 27227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "1adaf2a0e3394868ab8a1047c057b005"
format_stamp: "Formatted at 2026-08-12 09:44:27 on dist-test-slave-cbbz"
server_key: "f2d8c92f5d9c899f99a89146ed2d5a90"
server_key_iv: "58af92446efc51257fc0527e9b0862a3"
server_key_version: "encryptionkey@0"
I20260812 09:44:27.811007 27227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 09:44:27.813462 27243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:27.814634 27227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:44:27.814782 27227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "1adaf2a0e3394868ab8a1047c057b005"
format_stamp: "Formatted at 2026-08-12 09:44:27 on dist-test-slave-cbbz"
server_key: "f2d8c92f5d9c899f99a89146ed2d5a90"
server_key_iv: "58af92446efc51257fc0527e9b0862a3"
server_key_version: "encryptionkey@0"
I20260812 09:44:27.814898 27227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:27.819357 27227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:27.825636 27227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:27.825840 27227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:27.833022 27227 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46675
I20260812 09:44:27.833045 27303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46675 every 8 connection(s)
I20260812 09:44:27.833918 27227 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:27.836884 27304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:27.839630 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 27227
I20260812 09:44:27.839811 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:27.840118 3660 external_mini_cluster.cc:1428] Setting key d8f2e30577b6a3b5b382bb6cc70770ba
I20260812 09:44:27.843150 27304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005: Bootstrap starting.
I20260812 09:44:27.845419 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:27.841614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46061 (local address 127.0.0.1:46675)
0812 09:44:27.842048 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:27.842062 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:27.842097 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:44:27.842692 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:27.842707 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:27.842986 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:27.843285 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:27.844178 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:27.844187 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:27.844206 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:27.844227 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:27.844282 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:27.844302 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:27.844433 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:27.844547 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 09:44:27.844706 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":143,"threads_started":1}
I20260812 09:44:27.845991 27304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:27.846783 27304 log.cc:824] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:27.848223 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:27.849210 27304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005: No bootstrap required, opened a new log
I20260812 09:44:27.852229 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:27.850622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52501 (local address 127.0.0.1:46675)
0812 09:44:27.850744 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:27.850748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:27.850944 (+ 196us) server_negotiation.cc:407] Connection header received
0812 09:44:27.851063 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:27.851065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:27.851091 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:27.851153 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:27.851842 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:27.851845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:27.851846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:27.851856 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:27.851877 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:27.851886 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:27.851951 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:27.852062 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 09:44:27.852089 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 09:44:27.853996 27304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adaf2a0e3394868ab8a1047c057b005" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } }
I20260812 09:44:27.854250 27304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:27.854334 27304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1adaf2a0e3394868ab8a1047c057b005, State: Initialized, Role: FOLLOWER
I20260812 09:44:27.854770 27304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adaf2a0e3394868ab8a1047c057b005" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } }
I20260812 09:44:27.854899 27304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:27.854949 27304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:27.855015 27304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:27.856079 27304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adaf2a0e3394868ab8a1047c057b005" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } }
I20260812 09:44:27.856446 27304 leader_election.cc:302] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1adaf2a0e3394868ab8a1047c057b005; no voters:
I20260812 09:44:27.856710 27304 leader_election.cc:288] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:27.856828 27314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:27.857070 27314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [term 1 LEADER]: Becoming Leader. State: Replica: 1adaf2a0e3394868ab8a1047c057b005, State: Running, Role: LEADER
I20260812 09:44:27.857393 27314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adaf2a0e3394868ab8a1047c057b005" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } }
I20260812 09:44:27.857966 27304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:27.859266 27318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1adaf2a0e3394868ab8a1047c057b005. Latest consensus state: current_term: 1 leader_uuid: "1adaf2a0e3394868ab8a1047c057b005" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adaf2a0e3394868ab8a1047c057b005" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } } }
I20260812 09:44:27.859306 27317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1adaf2a0e3394868ab8a1047c057b005" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adaf2a0e3394868ab8a1047c057b005" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } } }
I20260812 09:44:27.859459 27318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:27.859481 27317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:27.860435 27325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:27.862974 27325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:27.868546 27325 catalog_manager.cc:1347] Generated new cluster ID: 074aac5fc96942c5a3b9f483e1dc0186
I20260812 09:44:27.868616 27325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:27.885008 27325 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:27.885331 27325 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:27.899592 27325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1adaf2a0e3394868ab8a1047c057b005: Generated new TSK 0
I20260812 09:44:27.900463 27325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:27.906073 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 27227
2026-08-12T09:44:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1483 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:27 dist-test-slave-cbbz krb5kdc[27340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:27 dist-test-slave-cbbz krb5kdc[27340](info): set up 2 sockets
Aug 12 09:44:27 dist-test-slave-cbbz krb5kdc[27340](info): commencing operation
krb5kdc: starting...
W20260812 09:44:29.020025 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:29 dist-test-slave-cbbz krb5kdc[27340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:29.144868 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:29.256717 27356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:29.256999 27356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:29.257087 27356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:29.261022 27356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:29.261113 27356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:29.261188 27356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:29.261226 27356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:29.261257 27356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:29.266368 27356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:29.267390 27356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:29.268412 27356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:29.276500 27365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:29.276503 27362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:29.276547 27363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:29.276835 27356 server_base.cc:1034] running on GCE node
I20260812 09:44:29.277371 27356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:29.278344 27356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:29.279531 27356 hybrid_clock.cc:648] HybridClock initialized: now 1786527869279517 us; error 36 us; skew 500 ppm
Aug 12 09:44:29 dist-test-slave-cbbz krb5kdc[27340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527869, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:29.282300 27356 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:29.283128 27356 webserver.cc:460] Webserver started at http://127.0.0.1:42157/ using document root <none> and password file <none>
I20260812 09:44:29.283667 27356 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:29.283766 27356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:29.284000 27356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:29.286018 27356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "ddd63f5d6b594ea0950cc0f758e0b534"
format_stamp: "Formatted at 2026-08-12 09:44:29 on dist-test-slave-cbbz"
server_key: "f1555c267869e27952e7a65c839c135b"
server_key_iv: "6a5d22959b57a876c74aef7dfc125ca6"
server_key_version: "encryptionkey@0"
I20260812 09:44:29.286618 27356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "ddd63f5d6b594ea0950cc0f758e0b534"
format_stamp: "Formatted at 2026-08-12 09:44:29 on dist-test-slave-cbbz"
server_key: "f1555c267869e27952e7a65c839c135b"
server_key_iv: "6a5d22959b57a876c74aef7dfc125ca6"
server_key_version: "encryptionkey@0"
I20260812 09:44:29.290215 27356 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 09:44:29.293340 27374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:29.294929 27356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:44:29.295060 27356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "ddd63f5d6b594ea0950cc0f758e0b534"
format_stamp: "Formatted at 2026-08-12 09:44:29 on dist-test-slave-cbbz"
server_key: "f1555c267869e27952e7a65c839c135b"
server_key_iv: "6a5d22959b57a876c74aef7dfc125ca6"
server_key_version: "encryptionkey@0"
I20260812 09:44:29.295248 27356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:29.314496 27356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:29.317231 27356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:29.317377 27356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:29.324990 27356 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37121
I20260812 09:44:29.325001 27436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37121 every 8 connection(s)
I20260812 09:44:29.325918 27356 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:29.328784 27437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:29.331555 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 27356
I20260812 09:44:29.331701 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:29.332012 3660 external_mini_cluster.cc:1428] Setting key db7f760c5243c85378cd8c76a9b63971
I20260812 09:44:29.334839 27437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534: Bootstrap starting.
I20260812 09:44:29.337347 27437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:29.338268 27437 log.cc:824] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:29.340783 27437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534: No bootstrap required, opened a new log
Aug 12 09:44:29 dist-test-slave-cbbz krb5kdc[27340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 09:44:29.344308 27437 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd63f5d6b594ea0950cc0f758e0b534" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } }
I20260812 09:44:29.344555 27437 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:29.344609 27437 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd63f5d6b594ea0950cc0f758e0b534, State: Initialized, Role: FOLLOWER
I20260812 09:44:29.345134 27437 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd63f5d6b594ea0950cc0f758e0b534" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } }
I20260812 09:44:29.345295 27437 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:29.345367 27437 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:29.345477 27437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:29.346678 27437 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd63f5d6b594ea0950cc0f758e0b534" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } }
I20260812 09:44:29.347023 27437 leader_election.cc:302] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddd63f5d6b594ea0950cc0f758e0b534; no voters:
I20260812 09:44:29.347246 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:29.333490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47217 (local address 127.0.0.1:37121)
0812 09:44:29.333944 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:29.333959 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:29.333994 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:44:29.334585 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:29.334601 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:29.334957 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:29.335277 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:29.336322 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:29.337298 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:29.338509 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:29.339179 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:29.339301 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:29.342119 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:29.342125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:29.342138 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:29.342164 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:29.343863 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:29.344370 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:29.344376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:29.344380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:29.344498 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:29.344804 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:29.344807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:29.344809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:29.345312 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:29.345544 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:29.345862 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 09:44:29.346198 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":138,"threads_started":1}
I20260812 09:44:29.347370 27437 leader_election.cc:288] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:29.347493 27444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:29.347757 27444 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [term 1 LEADER]: Becoming Leader. State: Replica: ddd63f5d6b594ea0950cc0f758e0b534, State: Running, Role: LEADER
I20260812 09:44:29.348060 27444 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd63f5d6b594ea0950cc0f758e0b534" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } }
I20260812 09:44:29.348702 27437 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:29.349303 27445 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddd63f5d6b594ea0950cc0f758e0b534. Latest consensus state: current_term: 1 leader_uuid: "ddd63f5d6b594ea0950cc0f758e0b534" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd63f5d6b594ea0950cc0f758e0b534" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } } }
I20260812 09:44:29.349416 27445 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:29.350076 27448 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddd63f5d6b594ea0950cc0f758e0b534" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd63f5d6b594ea0950cc0f758e0b534" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37121 } } }
I20260812 09:44:29.350167 27448 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:29.350971 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:29.351276 27452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:29.354607 27452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:29.360718 27452 catalog_manager.cc:1347] Generated new cluster ID: 2b514fcae80e4d22a9ea10d7a3981e00
I20260812 09:44:29.360790 27452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:29.361440 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:29.353086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55953 (local address 127.0.0.1:37121)
0812 09:44:29.353197 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:29.353201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:29.353437 (+ 236us) server_negotiation.cc:407] Connection header received
0812 09:44:29.353560 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:29.353565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:29.353612 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:29.353708 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:29.354347 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:29.354953 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:29.355770 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:29.356005 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:29.356113 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:29.357756 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:29.357760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:29.357763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:29.357787 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:29.359506 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:29.359989 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:29.359992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:29.359994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:29.360081 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:29.360460 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:29.360464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:29.360466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:29.360684 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:29.360786 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:29.361155 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 09:44:29.361286 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 09:44:29.382164 27452 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:29.383432 27452 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:29.389302 27452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddd63f5d6b594ea0950cc0f758e0b534: Generated new TSK 0
I20260812 09:44:29.389959 27452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:29.397279 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 27356
2026-08-12T09:44:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:29 dist-test-slave-cbbz krb5kdc[27473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:29 dist-test-slave-cbbz krb5kdc[27473](info): set up 2 sockets
Aug 12 09:44:29 dist-test-slave-cbbz krb5kdc[27473](info): commencing operation
krb5kdc: starting...
W20260812 09:44:30.517968 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:30 dist-test-slave-cbbz krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:30.643114 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:30.755586 27488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:30.755854 27488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:30.755895 27488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:30.759871 27488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:30.759933 27488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:30.759951 27488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:30.759967 27488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:30.759981 27488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:30.765164 27488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:30.766140 27488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:30.767144 27488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:30.775607 27494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:30.775606 27496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:30.775655 27493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:30.776049 27488 server_base.cc:1034] running on GCE node
I20260812 09:44:30.776687 27488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:30.777841 27488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:30.779006 27488 hybrid_clock.cc:648] HybridClock initialized: now 1786527870778985 us; error 33 us; skew 500 ppm
Aug 12 09:44:30 dist-test-slave-cbbz krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527870, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:30.782006 27488 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:30.782923 27488 webserver.cc:460] Webserver started at http://127.0.0.1:38703/ using document root <none> and password file <none>
I20260812 09:44:30.783495 27488 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:30.783557 27488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:30.783774 27488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:30.785897 27488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "2212582c4bb3451a9de823a998e599be"
format_stamp: "Formatted at 2026-08-12 09:44:30 on dist-test-slave-cbbz"
server_key: "13f2c5062d8201afe630065336e17650"
server_key_iv: "2f821f30cba6b2322a8e45c98d2e31d2"
server_key_version: "encryptionkey@0"
I20260812 09:44:30.786506 27488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "2212582c4bb3451a9de823a998e599be"
format_stamp: "Formatted at 2026-08-12 09:44:30 on dist-test-slave-cbbz"
server_key: "13f2c5062d8201afe630065336e17650"
server_key_iv: "2f821f30cba6b2322a8e45c98d2e31d2"
server_key_version: "encryptionkey@0"
I20260812 09:44:30.790279 27488 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260812 09:44:30.792719 27504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:30.793979 27488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:30.794204 27488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "2212582c4bb3451a9de823a998e599be"
format_stamp: "Formatted at 2026-08-12 09:44:30 on dist-test-slave-cbbz"
server_key: "13f2c5062d8201afe630065336e17650"
server_key_iv: "2f821f30cba6b2322a8e45c98d2e31d2"
server_key_version: "encryptionkey@0"
I20260812 09:44:30.794327 27488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:30.825443 27488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:30.828995 27488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:30.829229 27488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:30.836838 27488 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35353
I20260812 09:44:30.836869 27570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35353 every 8 connection(s)
I20260812 09:44:30.837829 27488 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:30.840646 27571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:30.840837 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 27488
I20260812 09:44:30.840999 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:30.841305 3660 external_mini_cluster.cc:1428] Setting key 39d8ef2c07a82b85cc1a2c791ccb5c7a
I20260812 09:44:30.848065 27571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be: Bootstrap starting.
I20260812 09:44:30.850770 27571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:30.851475 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:30.842819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45471 (local address 127.0.0.1:35353)
0812 09:44:30.843267 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:30.843286 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:30.843330 (+ 44us) server_negotiation.cc:407] Connection header received
0812 09:44:30.844084 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:30.844108 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:30.844595 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:30.844974 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:30.845384 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:30.846851 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:30.847772 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:30.848482 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:30.848641 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:30.849506 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:30.849513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:30.849528 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:30.850010 (+ 482us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:30.850076 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:30.850105 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:30.850218 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:30.850307 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 09:44:30.850507 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":124,"threads_started":1}
I20260812 09:44:30.851679 27571 log.cc:824] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:30.853750 27571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be: No bootstrap required, opened a new log
I20260812 09:44:30.854472 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:30.858196 27571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2212582c4bb3451a9de823a998e599be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } }
I20260812 09:44:30.858469 27571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:30.858548 27571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2212582c4bb3451a9de823a998e599be, State: Initialized, Role: FOLLOWER
I20260812 09:44:30.859007 27571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2212582c4bb3451a9de823a998e599be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } }
I20260812 09:44:30.859163 27571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:30.859231 27571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:30.859333 27571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:30.860433 27571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2212582c4bb3451a9de823a998e599be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } }
I20260812 09:44:30.860734 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:30.856556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57933 (local address 127.0.0.1:35353)
0812 09:44:30.856700 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:30.856706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:30.856827 (+ 121us) server_negotiation.cc:407] Connection header received
0812 09:44:30.856925 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:30.856928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:30.856976 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:30.857067 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:30.857271 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:30.858012 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:30.858967 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:30.859332 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:30.859450 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:30.860294 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:30.860301 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:30.860307 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:30.860324 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:30.860358 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:30.860376 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:30.860471 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:30.860553 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 09:44:30.860609 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 09:44:30.860814 27571 leader_election.cc:302] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2212582c4bb3451a9de823a998e599be; no voters:
I20260812 09:44:30.861151 27571 leader_election.cc:288] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:30.861253 27582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:30.861478 27582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [term 1 LEADER]: Becoming Leader. State: Replica: 2212582c4bb3451a9de823a998e599be, State: Running, Role: LEADER
I20260812 09:44:30.861791 27582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2212582c4bb3451a9de823a998e599be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } }
I20260812 09:44:30.862444 27571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:30.862931 27582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2212582c4bb3451a9de823a998e599be. Latest consensus state: current_term: 1 leader_uuid: "2212582c4bb3451a9de823a998e599be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2212582c4bb3451a9de823a998e599be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } } }
I20260812 09:44:30.863060 27582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:30.863461 27591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:30.863446 27585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2212582c4bb3451a9de823a998e599be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2212582c4bb3451a9de823a998e599be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35353 } } }
I20260812 09:44:30.863547 27585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:30.866860 27591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:30.873284 27591 catalog_manager.cc:1347] Generated new cluster ID: 27a0516f81f74020a249875f2c6d98ff
I20260812 09:44:30.873364 27591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:30.893075 27591 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:30.894050 27591 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:30.902781 27591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2212582c4bb3451a9de823a998e599be: Generated new TSK 0
I20260812 09:44:30.903530 27591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:30.914273 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 27488
2026-08-12T09:44:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:30 dist-test-slave-cbbz krb5kdc[27607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:30 dist-test-slave-cbbz krb5kdc[27607](info): set up 2 sockets
Aug 12 09:44:30 dist-test-slave-cbbz krb5kdc[27607](info): commencing operation
krb5kdc: starting...
W20260812 09:44:32.030850 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:32 dist-test-slave-cbbz krb5kdc[27607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:32.158594 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:32.268688 27623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:32.268955 27623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:32.268996 27623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:32.273002 27623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:32.273056 27623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:32.273075 27623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:32.273092 27623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:32.273106 27623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:32.278285 27623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:32.279281 27623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:32.280447 27623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:32.288615 27631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:32.288760 27623 server_base.cc:1034] running on GCE node
W20260812 09:44:32.288599 27629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:32.288678 27628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:32.289412 27623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:32.290491 27623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:32.291687 27623 hybrid_clock.cc:648] HybridClock initialized: now 1786527872291671 us; error 30 us; skew 500 ppm
Aug 12 09:44:32 dist-test-slave-cbbz krb5kdc[27607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:32.294660 27623 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:32.295547 27623 webserver.cc:460] Webserver started at http://127.0.0.1:39677/ using document root <none> and password file <none>
I20260812 09:44:32.296089 27623 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:32.296206 27623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:32.296448 27623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:32.298424 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "c59e6fdf24fc4ce6a74851439c13638e"
format_stamp: "Formatted at 2026-08-12 09:44:32 on dist-test-slave-cbbz"
server_key: "e47db71f8748f13255bcad99df68b94b"
server_key_iv: "e2a49cde81edf8e8d4583b5d10020a94"
server_key_version: "encryptionkey@0"
I20260812 09:44:32.298974 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "c59e6fdf24fc4ce6a74851439c13638e"
format_stamp: "Formatted at 2026-08-12 09:44:32 on dist-test-slave-cbbz"
server_key: "e47db71f8748f13255bcad99df68b94b"
server_key_iv: "e2a49cde81edf8e8d4583b5d10020a94"
server_key_version: "encryptionkey@0"
I20260812 09:44:32.302603 27623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 09:44:32.304962 27640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:32.306089 27623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:44:32.306236 27623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "c59e6fdf24fc4ce6a74851439c13638e"
format_stamp: "Formatted at 2026-08-12 09:44:32 on dist-test-slave-cbbz"
server_key: "e47db71f8748f13255bcad99df68b94b"
server_key_iv: "e2a49cde81edf8e8d4583b5d10020a94"
server_key_version: "encryptionkey@0"
I20260812 09:44:32.306356 27623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:32.311980 27623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:32.315181 27623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:32.315377 27623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:32.322494 27623 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36203
I20260812 09:44:32.322521 27705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36203 every 8 connection(s)
I20260812 09:44:32.323367 27623 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:32.324602 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 27623
I20260812 09:44:32.324743 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:32.325031 3660 external_mini_cluster.cc:1428] Setting key ce579d35ad62db187f9687b3f5429361
I20260812 09:44:32.326246 27706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:32.329975 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:32.326388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49529 (local address 127.0.0.1:36203)
0812 09:44:32.326817 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:32.326832 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:32.326867 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:44:32.327475 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:32.327490 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:32.327746 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:32.328082 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:32.328842 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:32.328847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:32.328862 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:32.328873 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:32.328926 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:32.328943 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:32.329093 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:32.329162 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 09:44:32.329309 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":139,"threads_started":1}
I20260812 09:44:32.332604 27706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e: Bootstrap starting.
I20260812 09:44:32.332947 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:32.334966 27706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:32.335971 27706 log.cc:824] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:32.336760 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:32.334975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49751 (local address 127.0.0.1:36203)
0812 09:44:32.335103 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:32.335109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:32.335411 (+ 302us) server_negotiation.cc:407] Connection header received
0812 09:44:32.335534 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:32.335536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:32.335563 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:32.335621 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:32.336355 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:32.336361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:32.336368 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:32.336378 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:32.336401 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:32.336411 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:32.336500 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:32.336591 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 09:44:32.336628 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 09:44:32.338090 27706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e: No bootstrap required, opened a new log
I20260812 09:44:32.341872 27706 raft_consensus.cc:348] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59e6fdf24fc4ce6a74851439c13638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36203 } }
I20260812 09:44:32.342106 27706 raft_consensus.cc:374] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:32.342176 27706 raft_consensus.cc:729] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59e6fdf24fc4ce6a74851439c13638e, State: Initialized, Role: FOLLOWER
I20260812 09:44:32.342563 27706 consensus_queue.cc:260] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59e6fdf24fc4ce6a74851439c13638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36203 } }
I20260812 09:44:32.342718 27706 raft_consensus.cc:388] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:32.342806 27706 raft_consensus.cc:482] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:32.342906 27706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:32.343825 27706 raft_consensus.cc:504] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59e6fdf24fc4ce6a74851439c13638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36203 } }
I20260812 09:44:32.344213 27706 leader_election.cc:302] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c59e6fdf24fc4ce6a74851439c13638e; no voters:
I20260812 09:44:32.344550 27706 leader_election.cc:288] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:32.344645 27718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:32.344936 27718 raft_consensus.cc:686] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [term 1 LEADER]: Becoming Leader. State: Replica: c59e6fdf24fc4ce6a74851439c13638e, State: Running, Role: LEADER
I20260812 09:44:32.345414 27718 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59e6fdf24fc4ce6a74851439c13638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36203 } }
I20260812 09:44:32.345840 27706 sys_catalog.cc:564] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:32.347016 27719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59e6fdf24fc4ce6a74851439c13638e. Latest consensus state: current_term: 1 leader_uuid: "c59e6fdf24fc4ce6a74851439c13638e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59e6fdf24fc4ce6a74851439c13638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36203 } } }
I20260812 09:44:32.347141 27719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:32.347312 27720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c59e6fdf24fc4ce6a74851439c13638e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59e6fdf24fc4ce6a74851439c13638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36203 } } }
I20260812 09:44:32.347461 27720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:32.347573 27728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:32.350184 27728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:32.355468 27728 catalog_manager.cc:1347] Generated new cluster ID: eea051f9e5984c2ba704f6ceb711a9b3
I20260812 09:44:32.355532 27728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:32.379268 27728 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:32.379666 27728 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:32.391642 27728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c59e6fdf24fc4ce6a74851439c13638e: Generated new TSK 0
I20260812 09:44:32.392441 27728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:32.404510 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 27623
2026-08-12T09:44:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:32 dist-test-slave-cbbz krb5kdc[27742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:32 dist-test-slave-cbbz krb5kdc[27742](info): set up 2 sockets
Aug 12 09:44:32 dist-test-slave-cbbz krb5kdc[27742](info): commencing operation
krb5kdc: starting...
W20260812 09:44:33.522871 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:33 dist-test-slave-cbbz krb5kdc[27742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:33.647421 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:33.759020 27757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:33.759325 27757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:33.759395 27757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:33.763501 27757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:33.763592 27757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:33.763628 27757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:33.763686 27757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:33.763723 27757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:33.769006 27757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:33.770054 27757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:33.771055 27757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:33.779731 27766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:33.779731 27764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:33.779826 27757 server_base.cc:1034] running on GCE node
W20260812 09:44:33.779759 27768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:33.780735 27757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:33.781893 27757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:33.783087 27757 hybrid_clock.cc:648] HybridClock initialized: now 1786527873783069 us; error 29 us; skew 500 ppm
Aug 12 09:44:33 dist-test-slave-cbbz krb5kdc[27742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527873, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:33.789172 27757 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:33.790068 27757 webserver.cc:460] Webserver started at http://127.0.0.1:46603/ using document root <none> and password file <none>
I20260812 09:44:33.790647 27757 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:33.790757 27757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:33.791006 27757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:33.793241 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "ba76f416672c4c14aa8ba6665d797247"
format_stamp: "Formatted at 2026-08-12 09:44:33 on dist-test-slave-cbbz"
server_key: "30d583aa5b100a28dd73fdf7e175f6bb"
server_key_iv: "c428c432d194f2c0bc0e742197d13cfc"
server_key_version: "encryptionkey@0"
I20260812 09:44:33.793823 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "ba76f416672c4c14aa8ba6665d797247"
format_stamp: "Formatted at 2026-08-12 09:44:33 on dist-test-slave-cbbz"
server_key: "30d583aa5b100a28dd73fdf7e175f6bb"
server_key_iv: "c428c432d194f2c0bc0e742197d13cfc"
server_key_version: "encryptionkey@0"
I20260812 09:44:33.797461 27757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:33.800002 27775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:33.801285 27757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:44:33.801455 27757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "ba76f416672c4c14aa8ba6665d797247"
format_stamp: "Formatted at 2026-08-12 09:44:33 on dist-test-slave-cbbz"
server_key: "30d583aa5b100a28dd73fdf7e175f6bb"
server_key_iv: "c428c432d194f2c0bc0e742197d13cfc"
server_key_version: "encryptionkey@0"
I20260812 09:44:33.801580 27757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:33.818197 27757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:33.821193 27757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:33.821403 27757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:33.829209 27757 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34981
I20260812 09:44:33.829221 27837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34981 every 8 connection(s)
I20260812 09:44:33.830271 27757 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:33.833107 27838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:33.833385 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 27757
I20260812 09:44:33.833555 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:33.833909 3660 external_mini_cluster.cc:1428] Setting key 1affa980713a2002f759d7ddcb5fdc91
I20260812 09:44:33.839805 27838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247: Bootstrap starting.
I20260812 09:44:33.842335 27838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:33.843045 27838 log.cc:824] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:44:33 dist-test-slave-cbbz krb5kdc[27742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 09:44:33.845158 27838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247: No bootstrap required, opened a new log
I20260812 09:44:33.849128 27838 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba76f416672c4c14aa8ba6665d797247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } }
I20260812 09:44:33.849355 27838 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:33.849423 27838 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba76f416672c4c14aa8ba6665d797247, State: Initialized, Role: FOLLOWER
I20260812 09:44:33.849844 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:33.835450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:38885 (local address 127.0.0.1:34981)
0812 09:44:33.835871 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:33.835889 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:33.835933 (+ 44us) server_negotiation.cc:407] Connection header received
0812 09:44:33.836802 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:33.836826 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:33.837248 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:33.837610 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:33.838356 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:33.839812 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:33.840865 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:33.841538 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:33.841705 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:33.844567 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:33.844573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:33.844586 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:33.844616 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:33.846517 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:33.847194 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:33.847199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:33.847204 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:33.847316 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:33.847667 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:33.847670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:33.847671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:33.848047 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:33.848321 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:33.848610 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 09:44:33.848871 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
I20260812 09:44:33.849851 27838 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba76f416672c4c14aa8ba6665d797247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } }
I20260812 09:44:33.850060 27838 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:33.850124 27838 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:33.850237 27838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:33.851105 27838 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba76f416672c4c14aa8ba6665d797247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } }
I20260812 09:44:33.851435 27838 leader_election.cc:302] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba76f416672c4c14aa8ba6665d797247; no voters:
I20260812 09:44:33.851755 27838 leader_election.cc:288] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:33.851828 27843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:33.852085 27843 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [term 1 LEADER]: Becoming Leader. State: Replica: ba76f416672c4c14aa8ba6665d797247, State: Running, Role: LEADER
I20260812 09:44:33.852444 27843 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba76f416672c4c14aa8ba6665d797247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } }
I20260812 09:44:33.853037 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:33.853111 27838 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:33.853675 27845 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba76f416672c4c14aa8ba6665d797247. Latest consensus state: current_term: 1 leader_uuid: "ba76f416672c4c14aa8ba6665d797247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba76f416672c4c14aa8ba6665d797247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } } }
I20260812 09:44:33.853806 27845 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:33.854176 27844 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba76f416672c4c14aa8ba6665d797247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba76f416672c4c14aa8ba6665d797247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } } }
I20260812 09:44:33.854306 27844 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247 [sys.catalog]: This master's current role is: LEADER
W20260812 09:44:33.858094 27869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:33.858189 27869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:33.858245 27852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:33.859056 27852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:33.865025 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:33.855629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:52619 (local address 127.0.0.1:34981)
0812 09:44:33.855745 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:33.855749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:33.856283 (+ 534us) server_negotiation.cc:407] Connection header received
0812 09:44:33.856474 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:33.856478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:33.856523 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:33.856612 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:33.857492 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:33.858256 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:33.859349 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:33.859735 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:33.859858 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:33.861426 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:33.861429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:33.861431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:33.861453 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:33.862889 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:33.863456 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:33.863460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:33.863462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:33.863556 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:33.864003 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:33.864006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:33.864008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:33.864241 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:33.864401 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:33.864773 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 09:44:33.864874 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:44:33.865329 27852 catalog_manager.cc:1347] Generated new cluster ID: 06c233cfab5e4dd698bf5a7022648900
I20260812 09:44:33.865378 27852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:33.890468 27852 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:33.891415 27852 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:33.897625 27852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba76f416672c4c14aa8ba6665d797247: Generated new TSK 0
I20260812 09:44:33.898411 27852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:33.902602 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 27757
W20260812 09:44:33.910626 27738 connection.cc:535] client connection to 127.0.0.1:34981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34981): Connection reset by peer (error 104)
2026-08-12T09:44:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:33 dist-test-slave-cbbz krb5kdc[27879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:33 dist-test-slave-cbbz krb5kdc[27879](info): set up 2 sockets
Aug 12 09:44:33 dist-test-slave-cbbz krb5kdc[27879](info): commencing operation
krb5kdc: starting...
W20260812 09:44:35.025854 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:35 dist-test-slave-cbbz krb5kdc[27879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:35.152658 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:35.262714 27894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:35.262961 27894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:35.263002 27894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:35.267525 27894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:35.267588 27894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:35.267607 27894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:35.267649 27894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:35.267683 27894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:35.273034 27894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:35.274111 27894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:35.275156 27894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:35.283632 27900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:35.283623 27899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:35.283623 27902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:35.283753 27894 server_base.cc:1034] running on GCE node
I20260812 09:44:35.284461 27894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:35.285492 27894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:35.286684 27894 hybrid_clock.cc:648] HybridClock initialized: now 1786527875286657 us; error 38 us; skew 500 ppm
Aug 12 09:44:35 dist-test-slave-cbbz krb5kdc[27879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:35.289531 27894 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:35.290480 27894 webserver.cc:460] Webserver started at http://127.0.0.1:42365/ using document root <none> and password file <none>
I20260812 09:44:35.291038 27894 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:35.291131 27894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:35.291371 27894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:35.293452 27894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "b6020a01ab034d089c8566f3a01cf408"
format_stamp: "Formatted at 2026-08-12 09:44:35 on dist-test-slave-cbbz"
server_key: "09aa9a02d00a4486d897183d2cde3676"
server_key_iv: "d979b1c2faab47e644a2bdd450f3f672"
server_key_version: "encryptionkey@0"
I20260812 09:44:35.294018 27894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "b6020a01ab034d089c8566f3a01cf408"
format_stamp: "Formatted at 2026-08-12 09:44:35 on dist-test-slave-cbbz"
server_key: "09aa9a02d00a4486d897183d2cde3676"
server_key_iv: "d979b1c2faab47e644a2bdd450f3f672"
server_key_version: "encryptionkey@0"
I20260812 09:44:35.297717 27894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260812 09:44:35.300125 27909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:35.301334 27894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:35.301484 27894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "b6020a01ab034d089c8566f3a01cf408"
format_stamp: "Formatted at 2026-08-12 09:44:35 on dist-test-slave-cbbz"
server_key: "09aa9a02d00a4486d897183d2cde3676"
server_key_iv: "d979b1c2faab47e644a2bdd450f3f672"
server_key_version: "encryptionkey@0"
I20260812 09:44:35.301599 27894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:35.314713 27894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:35.317588 27894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:35.317797 27894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:35.326143 27894 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40207
I20260812 09:44:35.326169 27970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40207 every 8 connection(s)
I20260812 09:44:35.327083 27894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:35.329203 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 27894
I20260812 09:44:35.329334 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:35.329659 3660 external_mini_cluster.cc:1428] Setting key 2380b028fa206eacf2bd321706f41c5c
I20260812 09:44:35.330766 27971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:35.336611 27971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408: Bootstrap starting.
I20260812 09:44:35.338685 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:35.331076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:56995 (local address 127.0.0.1:40207)
0812 09:44:35.331522 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:35.331538 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:35.331574 (+ 36us) server_negotiation.cc:407] Connection header received
0812 09:44:35.332277 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:35.332296 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:35.332635 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:35.332971 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:35.333363 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:35.334358 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:35.335235 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:35.335748 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:35.335915 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:35.336918 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:35.336935 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:35.336947 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:35.337318 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:35.337369 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:35.337390 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:35.337489 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:35.337719 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 09:44:35.337881 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":137,"threads_started":1}
I20260812 09:44:35.339080 27971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:35.339802 27971 log.cc:824] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:35.341676 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:35.342005 27971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408: No bootstrap required, opened a new log
I20260812 09:44:35.346333 27971 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6020a01ab034d089c8566f3a01cf408" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260812 09:44:35.346606 27971 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:35.346701 27971 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6020a01ab034d089c8566f3a01cf408, State: Initialized, Role: FOLLOWER
I20260812 09:44:35.347208 27971 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6020a01ab034d089c8566f3a01cf408" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260812 09:44:35.347376 27971 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:35.347463 27971 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:35.347595 27971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:35.348008 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:35.343743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:58167 (local address 127.0.0.1:40207)
0812 09:44:35.343853 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:35.343857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:35.344034 (+ 177us) server_negotiation.cc:407] Connection header received
0812 09:44:35.344212 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:35.344214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:35.344266 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:35.344344 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:35.344646 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:35.345120 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:35.346216 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:35.346437 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:35.346520 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:35.347457 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:35.347461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:35.347463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:35.347479 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:35.347505 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:35.347516 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:35.347620 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:35.347806 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 09:44:35.347844 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:44:35.348747 27971 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6020a01ab034d089c8566f3a01cf408" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260812 09:44:35.349092 27971 leader_election.cc:302] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6020a01ab034d089c8566f3a01cf408; no voters:
I20260812 09:44:35.349396 27971 leader_election.cc:288] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:35.349503 27983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:35.349777 27983 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [term 1 LEADER]: Becoming Leader. State: Replica: b6020a01ab034d089c8566f3a01cf408, State: Running, Role: LEADER
I20260812 09:44:35.350106 27983 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6020a01ab034d089c8566f3a01cf408" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260812 09:44:35.350679 27971 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:35.351642 27984 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6020a01ab034d089c8566f3a01cf408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6020a01ab034d089c8566f3a01cf408" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } } }
I20260812 09:44:35.351818 27984 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:35.352229 27985 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6020a01ab034d089c8566f3a01cf408. Latest consensus state: current_term: 1 leader_uuid: "b6020a01ab034d089c8566f3a01cf408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6020a01ab034d089c8566f3a01cf408" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } } }
I20260812 09:44:35.352320 27985 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:35.352391 27993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:35.354990 27993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:35.360652 27993 catalog_manager.cc:1347] Generated new cluster ID: dd9721aebf8d478fb9bfd68b19ef0178
I20260812 09:44:35.360720 27993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:35.375193 27993 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:35.376060 27993 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:35.386695 27993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6020a01ab034d089c8566f3a01cf408: Generated new TSK 0
I20260812 09:44:35.387398 27993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:35.393522 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 27894
W20260812 09:44:35.401885 27876 connection.cc:535] client connection to 127.0.0.1:40207 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40207): Connection reset by peer (error 104)
2026-08-12T09:44:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:35 dist-test-slave-cbbz krb5kdc[28007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:35 dist-test-slave-cbbz krb5kdc[28007](info): set up 2 sockets
Aug 12 09:44:35 dist-test-slave-cbbz krb5kdc[28007](info): commencing operation
krb5kdc: starting...
W20260812 09:44:36.498615 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:36 dist-test-slave-cbbz krb5kdc[28007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:36.624003 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43081
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:36.749712 28026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:36.750074 28026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:36.750164 28026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:36.756572 28026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:36.756656 28026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:36.756695 28026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:36.756769 28026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:36.756807 28026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:36.764288 28026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:36.765434 28026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:36.766835 28026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:36.776206 28031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:36.776211 28034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:36.776216 28032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:36.776254 28026 server_base.cc:1034] running on GCE node
I20260812 09:44:36.776988 28026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:36.778059 28026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:36.779253 28026 hybrid_clock.cc:648] HybridClock initialized: now 1786527876779243 us; error 25 us; skew 500 ppm
Aug 12 09:44:36 dist-test-slave-cbbz krb5kdc[28007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:36.781984 28026 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:36.782842 28026 webserver.cc:460] Webserver started at http://127.0.0.1:37911/ using document root <none> and password file <none>
I20260812 09:44:36.783396 28026 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:36.783488 28026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:36.783717 28026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:36.785949 28026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "329048483c3a4b25bb073055de931617"
format_stamp: "Formatted at 2026-08-12 09:44:36 on dist-test-slave-cbbz"
server_key: "8841022060be8d29f0d2f40efdbfa1cf"
server_key_iv: "baef273bbfd5f73af75054ec6dee99fb"
server_key_version: "encryptionkey@0"
I20260812 09:44:36.786520 28026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "329048483c3a4b25bb073055de931617"
format_stamp: "Formatted at 2026-08-12 09:44:36 on dist-test-slave-cbbz"
server_key: "8841022060be8d29f0d2f40efdbfa1cf"
server_key_iv: "baef273bbfd5f73af75054ec6dee99fb"
server_key_version: "encryptionkey@0"
I20260812 09:44:36.790202 28026 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 09:44:36.792781 28042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:36.793988 28026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:36.794143 28026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "329048483c3a4b25bb073055de931617"
format_stamp: "Formatted at 2026-08-12 09:44:36 on dist-test-slave-cbbz"
server_key: "8841022060be8d29f0d2f40efdbfa1cf"
server_key_iv: "baef273bbfd5f73af75054ec6dee99fb"
server_key_version: "encryptionkey@0"
I20260812 09:44:36.794257 28026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:36.798794 28026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:36.801967 28026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:36.802163 28026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:36.809134 28026 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43081
I20260812 09:44:36.809154 28099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43081 every 8 connection(s)
I20260812 09:44:36.810035 28026 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:36.810586 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 28026
I20260812 09:44:36.810739 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:36.811038 3660 external_mini_cluster.cc:1428] Setting key a26b280a4a94a703daf8de24d7958be5
I20260812 09:44:36.813194 28100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:36.816390 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:36.812511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:52231 (local address 127.0.0.1:43081)
0812 09:44:36.812949 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:36.812965 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:36.813001 (+ 36us) server_negotiation.cc:407] Connection header received
0812 09:44:36.813694 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:36.813730 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:36.814012 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:36.814282 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:36.815120 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:36.815126 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:36.815137 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:36.815149 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:36.815221 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:36.815240 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:36.815351 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:36.815477 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 09:44:36.815659 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":138,"threads_started":1}
I20260812 09:44:36.819343 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:36.819846 28100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617: Bootstrap starting.
I20260812 09:44:36.822626 28100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:36.823395 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:36.821707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:53959 (local address 127.0.0.1:43081)
0812 09:44:36.821841 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:36.821846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:36.821986 (+ 140us) server_negotiation.cc:407] Connection header received
0812 09:44:36.822077 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:36.822080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:36.822112 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:36.822190 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:36.822999 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:36.823001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:36.823003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:36.823014 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:36.823038 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:36.823048 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:36.823118 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:36.823221 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 09:44:36.823252 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 09:44:36.823601 28100 log.cc:824] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:36.825524 28100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617: No bootstrap required, opened a new log
I20260812 09:44:36.828954 28100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "329048483c3a4b25bb073055de931617" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } }
I20260812 09:44:36.829195 28100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:36.829284 28100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329048483c3a4b25bb073055de931617, State: Initialized, Role: FOLLOWER
I20260812 09:44:36.829680 28100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "329048483c3a4b25bb073055de931617" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } }
I20260812 09:44:36.829826 28100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:36.829916 28100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:36.830013 28100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:36.830878 28100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "329048483c3a4b25bb073055de931617" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } }
I20260812 09:44:36.831207 28100 leader_election.cc:302] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 329048483c3a4b25bb073055de931617; no voters:
I20260812 09:44:36.831514 28100 leader_election.cc:288] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:36.831643 28111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:36.831867 28111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [term 1 LEADER]: Becoming Leader. State: Replica: 329048483c3a4b25bb073055de931617, State: Running, Role: LEADER
I20260812 09:44:36.832175 28111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "329048483c3a4b25bb073055de931617" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } }
I20260812 09:44:36.832754 28100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:36.833896 28112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "329048483c3a4b25bb073055de931617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "329048483c3a4b25bb073055de931617" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } } }
I20260812 09:44:36.834033 28112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:36.834223 28113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 329048483c3a4b25bb073055de931617. Latest consensus state: current_term: 1 leader_uuid: "329048483c3a4b25bb073055de931617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "329048483c3a4b25bb073055de931617" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } } }
I20260812 09:44:36.834442 28120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:36.834368 28113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:36.838064 28120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:36.843173 28120 catalog_manager.cc:1347] Generated new cluster ID: 3f42da7f5b6a4e0d9c1ce79c89aceeff
I20260812 09:44:36.843243 28120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:36.864034 28120 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:36.864405 28120 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:36.872951 28120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 329048483c3a4b25bb073055de931617: Generated new TSK 0
I20260812 09:44:36.873625 28120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:36.880398 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 28026
2026-08-12T09:44:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:36 dist-test-slave-cbbz krb5kdc[28137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:36 dist-test-slave-cbbz krb5kdc[28137](info): set up 2 sockets
Aug 12 09:44:36 dist-test-slave-cbbz krb5kdc[28137](info): commencing operation
krb5kdc: starting...
W20260812 09:44:37.993702 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:38 dist-test-slave-cbbz krb5kdc[28137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:38.115376 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:38.224769 28153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:38.225035 28153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:38.225126 28153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:38.229142 28153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:38.229197 28153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:38.229218 28153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:38.229234 28153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:38.229249 28153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:38.234509 28153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36567
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:38.235553 28153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:38.236637 28153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:38.244926 28159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:38.244954 28162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:38.244938 28160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:38.245352 28153 server_base.cc:1034] running on GCE node
I20260812 09:44:38.245922 28153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:38.246997 28153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:38.248208 28153 hybrid_clock.cc:648] HybridClock initialized: now 1786527878248178 us; error 41 us; skew 500 ppm
Aug 12 09:44:38 dist-test-slave-cbbz krb5kdc[28137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527878, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:38.251096 28153 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:38.251917 28153 webserver.cc:460] Webserver started at http://127.0.0.1:33847/ using document root <none> and password file <none>
I20260812 09:44:38.252501 28153 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:38.252573 28153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:38.252758 28153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:38.254812 28153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "814651c65a3648388d8373917a1cbb85"
format_stamp: "Formatted at 2026-08-12 09:44:38 on dist-test-slave-cbbz"
server_key: "7083f1e7c694662e390cbe762666d16c"
server_key_iv: "7596758c17a3ce927eee46ec1c1ad9bc"
server_key_version: "encryptionkey@0"
I20260812 09:44:38.255381 28153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "814651c65a3648388d8373917a1cbb85"
format_stamp: "Formatted at 2026-08-12 09:44:38 on dist-test-slave-cbbz"
server_key: "7083f1e7c694662e390cbe762666d16c"
server_key_iv: "7596758c17a3ce927eee46ec1c1ad9bc"
server_key_version: "encryptionkey@0"
I20260812 09:44:38.259191 28153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 09:44:38.261730 28173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:38.262956 28153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:44:38.263065 28153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "814651c65a3648388d8373917a1cbb85"
format_stamp: "Formatted at 2026-08-12 09:44:38 on dist-test-slave-cbbz"
server_key: "7083f1e7c694662e390cbe762666d16c"
server_key_iv: "7596758c17a3ce927eee46ec1c1ad9bc"
server_key_version: "encryptionkey@0"
I20260812 09:44:38.263157 28153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:38.273828 28153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:38.276870 28153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:38.277040 28153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:38.284621 28227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36567 every 8 connection(s)
I20260812 09:44:38.284617 28153 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36567
I20260812 09:44:38.285743 28153 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:38.288636 28228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:38.291069 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 28153
I20260812 09:44:38.291232 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:38.291579 3660 external_mini_cluster.cc:1428] Setting key 5aa9dbcdecbe4c041326945c0c4cfb46
I20260812 09:44:38.295143 28228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85: Bootstrap starting.
I20260812 09:44:38.297421 28228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:38.298152 28228 log.cc:824] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:38.300096 28228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85: No bootstrap required, opened a new log
Aug 12 09:44:38 dist-test-slave-cbbz krb5kdc[28137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 09:44:38.303306 28228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814651c65a3648388d8373917a1cbb85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } }
I20260812 09:44:38.303572 28228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:38.303664 28228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 814651c65a3648388d8373917a1cbb85, State: Initialized, Role: FOLLOWER
I20260812 09:44:38.304168 28228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814651c65a3648388d8373917a1cbb85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } }
I20260812 09:44:38.304334 28228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:38.304440 28228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:38.304543 28228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:38.305359 28228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814651c65a3648388d8373917a1cbb85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } }
I20260812 09:44:38.305720 28228 leader_election.cc:302] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 814651c65a3648388d8373917a1cbb85; no voters:
I20260812 09:44:38.306037 28228 leader_election.cc:288] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:38.306141 28234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:38.306452 28234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [term 1 LEADER]: Becoming Leader. State: Replica: 814651c65a3648388d8373917a1cbb85, State: Running, Role: LEADER
I20260812 09:44:38.307014 28234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814651c65a3648388d8373917a1cbb85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } }
I20260812 09:44:38.307396 28228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:38.308105 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:38.293163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:35911 (local address 127.0.0.1:36567)
0812 09:44:38.293618 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:38.293639 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:38.293700 (+ 61us) server_negotiation.cc:407] Connection header received
0812 09:44:38.294378 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:38.294395 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:38.294729 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:38.295115 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:38.295985 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:38.297090 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:38.298075 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:38.298595 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:38.298788 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:38.301595 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:38.301601 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:38.301613 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:38.301641 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:38.304245 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:38.304956 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:38.304962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:38.304975 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:38.305101 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:38.305502 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:38.305506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:38.305508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:38.305967 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:38.306258 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:38.306887 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 09:44:38.307109 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":132,"threads_started":1}
I20260812 09:44:38.308760 28236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 814651c65a3648388d8373917a1cbb85. Latest consensus state: current_term: 1 leader_uuid: "814651c65a3648388d8373917a1cbb85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814651c65a3648388d8373917a1cbb85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } } }
I20260812 09:44:38.308902 28236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:38.309252 28235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "814651c65a3648388d8373917a1cbb85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814651c65a3648388d8373917a1cbb85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } } }
I20260812 09:44:38.309340 28235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:38.311555 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 09:44:38.312780 28251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:38.312945 28251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:38.313155 28247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:38.314019 28247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:38.321043 28247 catalog_manager.cc:1347] Generated new cluster ID: 70ff43f2995544b388e20e5785d08e45
I20260812 09:44:38.321151 28247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:38.322984 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:38.313913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:35255 (local address 127.0.0.1:36567)
0812 09:44:38.314022 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:38.314028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:38.314229 (+ 201us) server_negotiation.cc:407] Connection header received
0812 09:44:38.314341 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:38.314344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:38.314396 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:38.314495 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:38.315364 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:38.316074 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:38.316994 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:38.317239 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:38.317360 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:38.319133 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:38.319137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:38.319140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:38.319166 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:38.320823 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:38.321464 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:38.321468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:38.321470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:38.321576 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:38.322038 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:38.322042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:38.322043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:38.322257 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:38.322438 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:38.322730 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 09:44:38.322832 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 09:44:38.338012 28247 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:38.338989 28247 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:38.343804 28247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 814651c65a3648388d8373917a1cbb85: Generated new TSK 0
I20260812 09:44:38.344581 28247 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:38.353767 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 28153
W20260812 09:44:38.363552 28135 connection.cc:535] client connection to 127.0.0.1:36567 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36567): Connection reset by peer (error 104)
2026-08-12T09:44:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:38 dist-test-slave-cbbz krb5kdc[28264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:38 dist-test-slave-cbbz krb5kdc[28264](info): set up 2 sockets
Aug 12 09:44:38 dist-test-slave-cbbz krb5kdc[28264](info): commencing operation
krb5kdc: starting...
W20260812 09:44:39.473937 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:39 dist-test-slave-cbbz krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:39.597306 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:39.709146 28281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:39.709434 28281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:39.709478 28281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:39.713568 28281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:39.713639 28281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:39.713662 28281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:39.713682 28281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:39.713701 28281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:39.719501 28281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33899
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:39.720716 28281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:39.721765 28281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:39.730712 28290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:39.730719 28286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:39.730767 28281 server_base.cc:1034] running on GCE node
W20260812 09:44:39.730726 28287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:39.731547 28281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:39.732760 28281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:39.733969 28281 hybrid_clock.cc:648] HybridClock initialized: now 1786527879733943 us; error 44 us; skew 500 ppm
Aug 12 09:44:39 dist-test-slave-cbbz krb5kdc[28264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:39.740718 28281 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:39.741724 28281 webserver.cc:460] Webserver started at http://127.0.0.1:40681/ using document root <none> and password file <none>
I20260812 09:44:39.742305 28281 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:39.742374 28281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:39.742555 28281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:39.744901 28281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "7c93dc62deb743c38c512cd5284bde37"
format_stamp: "Formatted at 2026-08-12 09:44:39 on dist-test-slave-cbbz"
server_key: "85953fec999b43349d9a9117ac27f7f4"
server_key_iv: "1eb2c762ed2180b4491c46a38850d163"
server_key_version: "encryptionkey@0"
I20260812 09:44:39.745530 28281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "7c93dc62deb743c38c512cd5284bde37"
format_stamp: "Formatted at 2026-08-12 09:44:39 on dist-test-slave-cbbz"
server_key: "85953fec999b43349d9a9117ac27f7f4"
server_key_iv: "1eb2c762ed2180b4491c46a38850d163"
server_key_version: "encryptionkey@0"
I20260812 09:44:39.749680 28281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 09:44:39.752463 28297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:39.753747 28281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 09:44:39.753890 28281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "7c93dc62deb743c38c512cd5284bde37"
format_stamp: "Formatted at 2026-08-12 09:44:39 on dist-test-slave-cbbz"
server_key: "85953fec999b43349d9a9117ac27f7f4"
server_key_iv: "1eb2c762ed2180b4491c46a38850d163"
server_key_version: "encryptionkey@0"
I20260812 09:44:39.753999 28281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:39.786367 28281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:39.789354 28281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:39.789620 28281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:39.797331 28281 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33899
I20260812 09:44:39.797346 28360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33899 every 8 connection(s)
I20260812 09:44:39.798332 28281 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:39.801036 28361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:39.803709 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 28281
I20260812 09:44:39.803877 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:39.804270 3660 external_mini_cluster.cc:1428] Setting key afbf15c6b3b1691eb7b0bb3d860dddde
I20260812 09:44:39.806973 28361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37: Bootstrap starting.
I20260812 09:44:39.809610 28361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:39.810398 28361 log.cc:824] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:39.812374 28361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37: No bootstrap required, opened a new log
I20260812 09:44:39.813148 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:39.805595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:55231 (local address 127.0.0.1:33899)
0812 09:44:39.806008 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:39.806024 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:39.806059 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:44:39.806675 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:39.806692 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:39.807020 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:39.807364 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:39.807675 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:39.808733 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:39.809935 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:39.810386 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:39.810552 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:39.811498 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:39.811503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:39.811515 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:39.811891 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:39.811940 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:39.811961 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:39.812101 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:39.812322 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 09:44:39.812481 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":125,"threads_started":1}
I20260812 09:44:39.815920 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:39.815944 28361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c93dc62deb743c38c512cd5284bde37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33899 } }
I20260812 09:44:39.816236 28361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:39.816294 28361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c93dc62deb743c38c512cd5284bde37, State: Initialized, Role: FOLLOWER
I20260812 09:44:39.816687 28361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c93dc62deb743c38c512cd5284bde37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33899 } }
I20260812 09:44:39.816807 28361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:39.816888 28361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:39.817005 28361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:39.817976 28361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c93dc62deb743c38c512cd5284bde37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33899 } }
I20260812 09:44:39.818346 28361 leader_election.cc:302] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c93dc62deb743c38c512cd5284bde37; no voters:
I20260812 09:44:39.818683 28361 leader_election.cc:288] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:39.818769 28372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:39.819150 28372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [term 1 LEADER]: Becoming Leader. State: Replica: 7c93dc62deb743c38c512cd5284bde37, State: Running, Role: LEADER
I20260812 09:44:39.819491 28372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c93dc62deb743c38c512cd5284bde37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33899 } }
I20260812 09:44:39.820037 28361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:39.820642 28372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c93dc62deb743c38c512cd5284bde37. Latest consensus state: current_term: 1 leader_uuid: "7c93dc62deb743c38c512cd5284bde37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c93dc62deb743c38c512cd5284bde37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33899 } } }
I20260812 09:44:39.820832 28372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:39.821115 28373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c93dc62deb743c38c512cd5284bde37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c93dc62deb743c38c512cd5284bde37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33899 } } }
I20260812 09:44:39.821218 28373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:39.821581 28381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:39.823328 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:39.818161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:48771 (local address 127.0.0.1:33899)
0812 09:44:39.818265 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:39.818268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:39.818506 (+ 238us) server_negotiation.cc:407] Connection header received
0812 09:44:39.818624 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:39.818626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:39.818690 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:39.818777 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:39.819150 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:39.819874 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:39.821107 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:39.821434 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:39.821512 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:39.822678 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:39.822684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:39.822688 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:39.822702 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:39.822730 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:39.822748 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:39.822822 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:39.823173 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 09:44:39.823208 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 09:44:39.824807 28381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:39.830767 28381 catalog_manager.cc:1347] Generated new cluster ID: 7ac9d2fab839408ba3a90ddb8ddc30bb
I20260812 09:44:39.830840 28381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:39.841303 28381 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:39.842193 28381 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:39.850149 28381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c93dc62deb743c38c512cd5284bde37: Generated new TSK 0
I20260812 09:44:39.850750 28381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:39.857270 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 28281
W20260812 09:44:39.866092 28261 connection.cc:535] client connection to 127.0.0.1:33899 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33899): Connection reset by peer (error 104)
2026-08-12T09:44:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:39 dist-test-slave-cbbz krb5kdc[28393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:39 dist-test-slave-cbbz krb5kdc[28393](info): set up 2 sockets
Aug 12 09:44:39 dist-test-slave-cbbz krb5kdc[28393](info): commencing operation
krb5kdc: starting...
W20260812 09:44:41.111964 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:41 dist-test-slave-cbbz krb5kdc[28393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 09:44:41.238864 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:41.351135 28411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:41.351379 28411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:41.351419 28411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:41.355314 28411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:41.355368 28411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:41.355388 28411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:41.355404 28411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:41.355419 28411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:41.360476 28411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39743
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:41.361397 28411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:41.362353 28411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:41.370843 28421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:41.370893 28417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:41.370896 28418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:41.371165 28411 server_base.cc:1034] running on GCE node
I20260812 09:44:41.371835 28411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:41.372979 28411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:41.374182 28411 hybrid_clock.cc:648] HybridClock initialized: now 1786527881374166 us; error 45 us; skew 500 ppm
Aug 12 09:44:41 dist-test-slave-cbbz krb5kdc[28393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:41.377022 28411 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 09:44:41.377831 28411 webserver.cc:460] Webserver started at http://127.0.0.1:36755/ using document root <none> and password file <none>
I20260812 09:44:41.378322 28411 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:41.378381 28411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:41.378535 28411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:41.380560 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "aceef80cde96418ebceef940695c1fa1"
format_stamp: "Formatted at 2026-08-12 09:44:41 on dist-test-slave-cbbz"
server_key: "901f38b673000f3e755abff899367e87"
server_key_iv: "7ef649442911558448d91e95781b0223"
server_key_version: "encryptionkey@0"
I20260812 09:44:41.381072 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "aceef80cde96418ebceef940695c1fa1"
format_stamp: "Formatted at 2026-08-12 09:44:41 on dist-test-slave-cbbz"
server_key: "901f38b673000f3e755abff899367e87"
server_key_iv: "7ef649442911558448d91e95781b0223"
server_key_version: "encryptionkey@0"
I20260812 09:44:41.384629 28411 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 09:44:41.386922 28430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:41.388125 28411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:41.388298 28411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "aceef80cde96418ebceef940695c1fa1"
format_stamp: "Formatted at 2026-08-12 09:44:41 on dist-test-slave-cbbz"
server_key: "901f38b673000f3e755abff899367e87"
server_key_iv: "7ef649442911558448d91e95781b0223"
server_key_version: "encryptionkey@0"
I20260812 09:44:41.388381 28411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:41.392663 28411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:41.395846 28411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:41.395992 28411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:41.402925 28411 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39743
I20260812 09:44:41.402997 28491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39743 every 8 connection(s)
I20260812 09:44:41.403827 28411 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:41.404527 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 28411
I20260812 09:44:41.404671 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:41.404951 3660 external_mini_cluster.cc:1428] Setting key ba35129c592a25145f7095d2b31c54ad
I20260812 09:44:41.406987 28492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:41.409937 28496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:41.406295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:59309 (local address 127.0.0.1:39743)
0812 09:44:41.406728 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:41.406742 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:41.406775 (+ 33us) server_negotiation.cc:407] Connection header received
0812 09:44:41.407450 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:41.407467 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:41.407731 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:41.408051 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:41.408821 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:41.408826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:41.408848 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:41.408858 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:41.408907 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:41.408925 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:41.409072 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:41.409139 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 09:44:41.409299 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":139,"threads_started":1}
I20260812 09:44:41.412786 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:41.413422 28492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1: Bootstrap starting.
I20260812 09:44:41.416232 28492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:41.416766 28496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:41.415071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.249:35483 (local address 127.0.0.1:39743)
0812 09:44:41.415201 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:41.415206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:41.415464 (+ 258us) server_negotiation.cc:407] Connection header received
0812 09:44:41.415571 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:41.415573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:41.415600 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:41.415663 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:41.416363 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:41.416369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:41.416372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:41.416384 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:41.416415 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:41.416425 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:41.416495 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:41.416614 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 09:44:41.416642 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 09:44:41.417176 28492 log.cc:824] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:41.419032 28492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1: No bootstrap required, opened a new log
I20260812 09:44:41.422456 28492 raft_consensus.cc:348] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aceef80cde96418ebceef940695c1fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260812 09:44:41.422703 28492 raft_consensus.cc:374] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:41.422786 28492 raft_consensus.cc:729] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aceef80cde96418ebceef940695c1fa1, State: Initialized, Role: FOLLOWER
I20260812 09:44:41.423192 28492 consensus_queue.cc:260] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aceef80cde96418ebceef940695c1fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260812 09:44:41.423342 28492 raft_consensus.cc:388] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:41.423436 28492 raft_consensus.cc:482] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:41.423542 28492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:41.424474 28492 raft_consensus.cc:504] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aceef80cde96418ebceef940695c1fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260812 09:44:41.424829 28492 leader_election.cc:302] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aceef80cde96418ebceef940695c1fa1; no voters:
I20260812 09:44:41.425124 28492 leader_election.cc:288] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:41.425204 28505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:41.425393 28505 raft_consensus.cc:686] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [term 1 LEADER]: Becoming Leader. State: Replica: aceef80cde96418ebceef940695c1fa1, State: Running, Role: LEADER
I20260812 09:44:41.425680 28505 consensus_queue.cc:237] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aceef80cde96418ebceef940695c1fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260812 09:44:41.426389 28492 sys_catalog.cc:564] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:41.427569 28507 sys_catalog.cc:455] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aceef80cde96418ebceef940695c1fa1. Latest consensus state: current_term: 1 leader_uuid: "aceef80cde96418ebceef940695c1fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aceef80cde96418ebceef940695c1fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } } }
I20260812 09:44:41.427697 28507 sys_catalog.cc:458] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:41.427940 28508 sys_catalog.cc:455] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aceef80cde96418ebceef940695c1fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aceef80cde96418ebceef940695c1fa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } } }
I20260812 09:44:41.428030 28508 sys_catalog.cc:458] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:41.428213 28515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:41.431460 28515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:41.437304 28515 catalog_manager.cc:1347] Generated new cluster ID: 352d88894cd64f7f9bd74b71aa89dd26
I20260812 09:44:41.437374 28515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:41.467061 28515 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:41.467427 28515 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:41.473337 28515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aceef80cde96418ebceef940695c1fa1: Generated new TSK 0
I20260812 09:44:41.474059 28515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:41.480386 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 28411
2026-08-12T09:44:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1625 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:41 dist-test-slave-cbbz krb5kdc[28532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:41 dist-test-slave-cbbz krb5kdc[28532](info): set up 2 sockets
Aug 12 09:44:41 dist-test-slave-cbbz krb5kdc[28532](info): commencing operation
krb5kdc: starting...
W20260812 09:44:42.594862 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:42 dist-test-slave-cbbz krb5kdc[28532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:42.723948 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35427
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:35427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:42.837247 28548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:42.837545 28548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:42.837636 28548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:42.841712 28548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:42.841799 28548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:42.841861 28548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:42.841920 28548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:42.841961 28548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:42.847343 28548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:35427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:35427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:42.848541 28548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:42.849666 28548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:42.858180 28553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:42.858244 28548 server_base.cc:1034] running on GCE node
W20260812 09:44:42.858229 28558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:42.858245 28554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:42.859017 28548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:42.860183 28548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:42.861404 28548 hybrid_clock.cc:648] HybridClock initialized: now 1786527882861382 us; error 31 us; skew 500 ppm
Aug 12 09:44:42 dist-test-slave-cbbz krb5kdc[28532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527882, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:42.864051 28548 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:42.864928 28548 webserver.cc:460] Webserver started at http://127.3.147.62:34929/ using document root <none> and password file <none>
I20260812 09:44:42.865476 28548 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:42.865567 28548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:42.865805 28548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:42.867805 28548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "5b6f9ea943054874a5841302ad8658df"
format_stamp: "Formatted at 2026-08-12 09:44:42 on dist-test-slave-cbbz"
server_key: "d0d2b01757da5f82b15c8bc7ef950750"
server_key_iv: "fdd276a1e768f5d6a696502178e01f9a"
server_key_version: "encryptionkey@0"
I20260812 09:44:42.868404 28548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "5b6f9ea943054874a5841302ad8658df"
format_stamp: "Formatted at 2026-08-12 09:44:42 on dist-test-slave-cbbz"
server_key: "d0d2b01757da5f82b15c8bc7ef950750"
server_key_iv: "fdd276a1e768f5d6a696502178e01f9a"
server_key_version: "encryptionkey@0"
I20260812 09:44:42.872042 28548 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 09:44:42.874418 28566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:42.875604 28548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:42.875751 28548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "5b6f9ea943054874a5841302ad8658df"
format_stamp: "Formatted at 2026-08-12 09:44:42 on dist-test-slave-cbbz"
server_key: "d0d2b01757da5f82b15c8bc7ef950750"
server_key_iv: "fdd276a1e768f5d6a696502178e01f9a"
server_key_version: "encryptionkey@0"
I20260812 09:44:42.875866 28548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:42.887042 28548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:42.889983 28548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:42.890183 28548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:42.897619 28548 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:35427
I20260812 09:44:42.897648 28629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:35427 every 8 connection(s)
I20260812 09:44:42.898558 28548 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:42.900382 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 28548
I20260812 09:44:42.900530 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:42.900828 3660 external_mini_cluster.cc:1428] Setting key faf89a3d7df075a89b76a1edc5bf2d7a
I20260812 09:44:42.901539 28631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:42.908438 28631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df: Bootstrap starting.
Aug 12 09:44:42 dist-test-slave-cbbz krb5kdc[28532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:42.910991 28631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:42.911952 28631 log.cc:824] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:42.913849 28631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df: No bootstrap required, opened a new log
I20260812 09:44:42.915902 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:42.902221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58927 (local address 127.3.147.62:35427)
0812 09:44:42.902640 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:42.902659 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:42.902702 (+ 43us) server_negotiation.cc:407] Connection header received
0812 09:44:42.903540 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:42.903562 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:42.904008 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:42.904374 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:42.905235 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:42.906748 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:42.907610 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:42.908327 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:42.908507 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:42.911146 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:42.911155 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:42.911172 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:42.911206 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:42.913018 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:42.913543 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:42.913548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:42.913552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:42.913661 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:42.914053 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:42.914061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:42.914062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:42.914436 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:42.914650 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:42.914976 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 09:44:42.915195 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":112,"threads_started":1}
I20260812 09:44:42.917712 28631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6f9ea943054874a5841302ad8658df" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35427 } }
I20260812 09:44:42.918009 28631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:42.918074 28631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b6f9ea943054874a5841302ad8658df, State: Initialized, Role: FOLLOWER
I20260812 09:44:42.918596 28631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6f9ea943054874a5841302ad8658df" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35427 } }
I20260812 09:44:42.918753 28631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:42.918823 28631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:42.918923 28631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:42.919086 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:42.919871 28631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6f9ea943054874a5841302ad8658df" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35427 } }
I20260812 09:44:42.920262 28631 leader_election.cc:302] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b6f9ea943054874a5841302ad8658df; no voters:
I20260812 09:44:42.920629 28631 leader_election.cc:288] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:42.920707 28642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:42.920907 28642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [term 1 LEADER]: Becoming Leader. State: Replica: 5b6f9ea943054874a5841302ad8658df, State: Running, Role: LEADER
I20260812 09:44:42.921190 28642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6f9ea943054874a5841302ad8658df" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35427 } }
I20260812 09:44:42.921940 28631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:42.922362 28643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b6f9ea943054874a5841302ad8658df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6f9ea943054874a5841302ad8658df" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35427 } } }
I20260812 09:44:42.922369 28642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b6f9ea943054874a5841302ad8658df. Latest consensus state: current_term: 1 leader_uuid: "5b6f9ea943054874a5841302ad8658df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6f9ea943054874a5841302ad8658df" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 35427 } } }
I20260812 09:44:42.922565 28643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:42.922605 28642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:42.923108 28647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:42.926905 28647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:42.930962 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:42.921522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35539 (local address 127.3.147.62:35427)
0812 09:44:42.921624 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:42.921628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:42.922093 (+ 465us) server_negotiation.cc:407] Connection header received
0812 09:44:42.922137 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:42.922139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:42.922175 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:42.922268 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:42.923433 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:42.924211 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:42.925320 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:42.925651 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:42.925768 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:42.927426 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:42.927430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:42.927433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:42.927457 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:42.929193 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:42.929660 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:42.929664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:42.929665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:42.929749 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:42.930073 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:42.930076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:42.930078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:42.930252 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:42.930398 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:42.930703 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 09:44:42.930797 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 09:44:42.933552 28647 catalog_manager.cc:1347] Generated new cluster ID: ac87a2a845444c7f85bc16c16dd98327
I20260812 09:44:42.933633 28647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:42.943387 28647 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:42.944550 28647 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:42.956583 28647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b6f9ea943054874a5841302ad8658df: Generated new TSK 0
I20260812 09:44:42.957362 28647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:42.967701 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 28548
W20260812 09:44:42.978188 28530 connection.cc:535] client connection to 127.3.147.62:35427 recv error: Network error: failed to read from TLS socket (remote: 127.3.147.62:35427): Connection reset by peer (error 104)
2026-08-12T09:44:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:43 dist-test-slave-cbbz krb5kdc[28669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:43 dist-test-slave-cbbz krb5kdc[28669](info): set up 2 sockets
Aug 12 09:44:43 dist-test-slave-cbbz krb5kdc[28669](info): commencing operation
krb5kdc: starting...
W20260812 09:44:44.082201 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:44 dist-test-slave-cbbz krb5kdc[28669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:44.206916 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38909
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:34629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:38909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:44.317998 28685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:44.318264 28685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:44.318306 28685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:44.322378 28685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:44.322433 28685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:44.322453 28685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:44.322515 28685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:44.322551 28685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:44.330366 28685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:34629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:38909
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:38909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:44.332475 28685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:44.333551 28685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:44.342724 28690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:44.342706 28693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:44.342741 28691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:44.342922 28685 server_base.cc:1034] running on GCE node
I20260812 09:44:44.343565 28685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:44.344739 28685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:44.345937 28685 hybrid_clock.cc:648] HybridClock initialized: now 1786527884345918 us; error 30 us; skew 500 ppm
Aug 12 09:44:44 dist-test-slave-cbbz krb5kdc[28669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527884, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:44.348694 28685 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:44.349627 28685 webserver.cc:460] Webserver started at http://127.3.147.62:34407/ using document root <none> and password file <none>
I20260812 09:44:44.350216 28685 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:44.350314 28685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:44.350549 28685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:44.352802 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b"
format_stamp: "Formatted at 2026-08-12 09:44:44 on dist-test-slave-cbbz"
server_key: "91bf9ff947ad0880b5c1b362097653b5"
server_key_iv: "20514fd957417aa081a1831726b0def5"
server_key_version: "encryptionkey@0"
I20260812 09:44:44.353502 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b"
format_stamp: "Formatted at 2026-08-12 09:44:44 on dist-test-slave-cbbz"
server_key: "91bf9ff947ad0880b5c1b362097653b5"
server_key_iv: "20514fd957417aa081a1831726b0def5"
server_key_version: "encryptionkey@0"
I20260812 09:44:44.357836 28685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:44:44.360256 28702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:44.361476 28685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:44:44.361621 28685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b"
format_stamp: "Formatted at 2026-08-12 09:44:44 on dist-test-slave-cbbz"
server_key: "91bf9ff947ad0880b5c1b362097653b5"
server_key_iv: "20514fd957417aa081a1831726b0def5"
server_key_version: "encryptionkey@0"
I20260812 09:44:44.361732 28685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:44.391170 28685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:44.394381 28685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:44.394614 28685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:44.402145 28685 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:38909
I20260812 09:44:44.402165 28764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:38909 every 8 connection(s)
I20260812 09:44:44.403080 28685 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:44.403767 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 28685
I20260812 09:44:44.403910 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:44.404289 3660 external_mini_cluster.cc:1428] Setting key bb95b5d36d8722aa9feb9948235c799f
I20260812 09:44:44.406418 28765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:44.411914 28765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b: Bootstrap starting.
I20260812 09:44:44.413323 28768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:44.405705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36963 (local address 127.3.147.62:38909)
0812 09:44:44.406162 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:44.406177 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:44.406211 (+ 34us) server_negotiation.cc:407] Connection header received
0812 09:44:44.406902 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:44.406920 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:44.407242 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:44.407577 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:44.408052 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:44.409111 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:44.410017 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:44.410469 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:44.410673 (+ 204us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:44.411609 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:44.411614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:44.411632 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:44.412030 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:44.412094 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:44.412118 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:44.412331 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:44.412515 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 09:44:44.412661 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":152,"threads_started":1}
I20260812 09:44:44.414318 28765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:44.415222 28765 log.cc:824] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:44.416424 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:44.417403 28765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b: No bootstrap required, opened a new log
I20260812 09:44:44.422075 28765 raft_consensus.cc:348] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38909 } }
I20260812 09:44:44.422360 28765 raft_consensus.cc:374] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:44.422442 28765 raft_consensus.cc:729] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68d1ad34a8d4ddaaa7c46fd81a3764b, State: Initialized, Role: FOLLOWER
I20260812 09:44:44.422755 28768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:44.418664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60235 (local address 127.3.147.62:38909)
0812 09:44:44.418766 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:44.418769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:44.418902 (+ 133us) server_negotiation.cc:407] Connection header received
0812 09:44:44.419003 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:44.419005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:44.419056 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:44.419125 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:44.419423 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:44.419910 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:44.421041 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:44.421242 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:44.421320 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:44.422150 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:44.422153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:44.422155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:44.422166 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:44.422188 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:44.422198 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:44.422266 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:44.422541 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 09:44:44.422582 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:44:44.422935 28765 consensus_queue.cc:260] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38909 } }
I20260812 09:44:44.423107 28765 raft_consensus.cc:388] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:44.423185 28765 raft_consensus.cc:482] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:44.423287 28765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:44.424288 28765 raft_consensus.cc:504] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38909 } }
I20260812 09:44:44.424584 28765 leader_election.cc:302] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b68d1ad34a8d4ddaaa7c46fd81a3764b; no voters:
I20260812 09:44:44.424827 28765 leader_election.cc:288] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:44.424906 28776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:44.425179 28776 raft_consensus.cc:686] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [term 1 LEADER]: Becoming Leader. State: Replica: b68d1ad34a8d4ddaaa7c46fd81a3764b, State: Running, Role: LEADER
I20260812 09:44:44.425450 28776 consensus_queue.cc:237] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38909 } }
I20260812 09:44:44.426031 28765 sys_catalog.cc:564] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:44.427124 28777 sys_catalog.cc:455] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38909 } } }
I20260812 09:44:44.427249 28777 sys_catalog.cc:458] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:44.427181 28778 sys_catalog.cc:455] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b68d1ad34a8d4ddaaa7c46fd81a3764b. Latest consensus state: current_term: 1 leader_uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68d1ad34a8d4ddaaa7c46fd81a3764b" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 38909 } } }
I20260812 09:44:44.427310 28778 sys_catalog.cc:458] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:44.427596 28786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:44.430799 28786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:44.436594 28786 catalog_manager.cc:1347] Generated new cluster ID: 64818b17003a4bb09c10aaa75d82e652
I20260812 09:44:44.436666 28786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:44.446727 28786 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:44.447919 28786 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:44.456996 28786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b68d1ad34a8d4ddaaa7c46fd81a3764b: Generated new TSK 0
I20260812 09:44:44.457891 28786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:44.470858 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 28685
W20260812 09:44:44.479420 28667 connection.cc:535] client connection to 127.3.147.62:38909 recv error: Network error: failed to read from TLS socket (remote: 127.3.147.62:38909): Connection reset by peer (error 104)
2026-08-12T09:44:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:44 dist-test-slave-cbbz krb5kdc[28800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:44 dist-test-slave-cbbz krb5kdc[28800](info): set up 2 sockets
Aug 12 09:44:44 dist-test-slave-cbbz krb5kdc[28800](info): commencing operation
krb5kdc: starting...
W20260812 09:44:45.588104 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:45 dist-test-slave-cbbz krb5kdc[28800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:45.715339 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:45771
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:45771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:45.826503 28820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:45.826826 28820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:45.826905 28820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:45.830889 28820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:45.830987 28820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:45.831028 28820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:45.831087 28820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:45.831121 28820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:45.836315 28820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:45771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:45.837410 28820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:45.838449 28820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:45.846987 28828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:45.847165 28820 server_base.cc:1034] running on GCE node
W20260812 09:44:45.847046 28825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:45.846997 28826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:45.847821 28820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:45.848965 28820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:45.850131 28820 hybrid_clock.cc:648] HybridClock initialized: now 1786527885850109 us; error 33 us; skew 500 ppm
Aug 12 09:44:45 dist-test-slave-cbbz krb5kdc[28800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527885, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:45.857203 28820 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:45.858124 28820 webserver.cc:460] Webserver started at http://127.3.147.62:45519/ using document root <none> and password file <none>
I20260812 09:44:45.858678 28820 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:45.858741 28820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:45.858980 28820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:45.861053 28820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "8fb76453a33240ec92e1749fc4e0a450"
format_stamp: "Formatted at 2026-08-12 09:44:45 on dist-test-slave-cbbz"
server_key: "f77fd5fe448b4e17377f099529a579c2"
server_key_iv: "739da18b70865fb9f64ee44b8c81b768"
server_key_version: "encryptionkey@0"
I20260812 09:44:45.861609 28820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "8fb76453a33240ec92e1749fc4e0a450"
format_stamp: "Formatted at 2026-08-12 09:44:45 on dist-test-slave-cbbz"
server_key: "f77fd5fe448b4e17377f099529a579c2"
server_key_iv: "739da18b70865fb9f64ee44b8c81b768"
server_key_version: "encryptionkey@0"
I20260812 09:44:45.865216 28820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 09:44:45.867453 28836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:45.868665 28820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:45.868791 28820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "8fb76453a33240ec92e1749fc4e0a450"
format_stamp: "Formatted at 2026-08-12 09:44:45 on dist-test-slave-cbbz"
server_key: "f77fd5fe448b4e17377f099529a579c2"
server_key_iv: "739da18b70865fb9f64ee44b8c81b768"
server_key_version: "encryptionkey@0"
I20260812 09:44:45.868888 28820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:45.873337 28820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:45.875996 28820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:45.876214 28820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:45.883374 28820 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:45771
I20260812 09:44:45.883404 28896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:45771 every 8 connection(s)
I20260812 09:44:45.884332 28820 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:45.886978 28897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:45.892292 28897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450: Bootstrap starting.
I20260812 09:44:45.892387 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 28820
I20260812 09:44:45.892553 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:45.892827 3660 external_mini_cluster.cc:1428] Setting key dd55ffd46ea1643d1d5523bf038f53e8
I20260812 09:44:45.894817 28897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:45.895572 28897 log.cc:824] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:45.897967 28897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450: No bootstrap required, opened a new log
I20260812 09:44:45.898129 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:45.894160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57997 (local address 127.3.147.62:45771)
0812 09:44:45.894602 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:45.894623 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:45.894658 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:44:45.895327 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:45.895346 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:45.895614 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:45.895983 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:45.896789 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:45.896795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:45.896809 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:45.896821 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:45.896884 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:45.896910 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:45.897020 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:45.897108 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 09:44:45.897256 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":135,"threads_started":1}
I20260812 09:44:45.901181 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:45.901556 28897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb76453a33240ec92e1749fc4e0a450" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45771 } }
I20260812 09:44:45.901758 28897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:45.901846 28897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb76453a33240ec92e1749fc4e0a450, State: Initialized, Role: FOLLOWER
I20260812 09:44:45.902244 28897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb76453a33240ec92e1749fc4e0a450" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45771 } }
I20260812 09:44:45.902361 28897 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:45.902478 28897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:45.902609 28897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:45.903617 28897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb76453a33240ec92e1749fc4e0a450" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45771 } }
I20260812 09:44:45.904021 28897 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb76453a33240ec92e1749fc4e0a450; no voters:
I20260812 09:44:45.904413 28897 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:45.904613 28911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:45.904891 28911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [term 1 LEADER]: Becoming Leader. State: Replica: 8fb76453a33240ec92e1749fc4e0a450, State: Running, Role: LEADER
I20260812 09:44:45.905114 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:45.903393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32873 (local address 127.3.147.62:45771)
0812 09:44:45.903507 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:45.903512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:45.903704 (+ 192us) server_negotiation.cc:407] Connection header received
0812 09:44:45.903826 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:45.903829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:45.903857 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:45.903922 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:45.904649 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:45.904653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:45.904657 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:45.904670 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:45.904700 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:45.904716 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:45.904804 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:45.904969 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 09:44:45.905007 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 09:44:45.905268 28911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb76453a33240ec92e1749fc4e0a450" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45771 } }
I20260812 09:44:45.905683 28897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:45.906512 28911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb76453a33240ec92e1749fc4e0a450. Latest consensus state: current_term: 1 leader_uuid: "8fb76453a33240ec92e1749fc4e0a450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb76453a33240ec92e1749fc4e0a450" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45771 } } }
I20260812 09:44:45.906687 28911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:45.906987 28912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fb76453a33240ec92e1749fc4e0a450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb76453a33240ec92e1749fc4e0a450" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45771 } } }
I20260812 09:44:45.907094 28912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:45.907218 28921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:45.910394 28921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:45.915669 28921 catalog_manager.cc:1347] Generated new cluster ID: bd407950698d422e91b0bd62f8ee5c3b
I20260812 09:44:45.915763 28921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:45.934405 28921 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:45.934703 28921 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:45.942075 28921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fb76453a33240ec92e1749fc4e0a450: Generated new TSK 0
I20260812 09:44:45.942875 28921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:45.956823 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 28820
2026-08-12T09:44:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:46 dist-test-slave-cbbz krb5kdc[28936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:46 dist-test-slave-cbbz krb5kdc[28936](info): set up 2 sockets
Aug 12 09:44:46 dist-test-slave-cbbz krb5kdc[28936](info): commencing operation
krb5kdc: starting...
W20260812 09:44:47.080307 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:47 dist-test-slave-cbbz krb5kdc[28936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:47.207152 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:46527
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:46527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:47.317777 28951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:47.318054 28951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:47.318140 28951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:47.322237 28951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:47.322319 28951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:47.322355 28951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:47.322407 28951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:47.322440 28951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:47.327587 28951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:46527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:46527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:47.328681 28951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:47.329677 28951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:47.338243 28958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:47.338253 28960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:47.338258 28956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:47.338449 28951 server_base.cc:1034] running on GCE node
I20260812 09:44:47.339217 28951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:47.340380 28951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:47.341579 28951 hybrid_clock.cc:648] HybridClock initialized: now 1786527887341556 us; error 30 us; skew 500 ppm
Aug 12 09:44:47 dist-test-slave-cbbz krb5kdc[28936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527887, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:47.344285 28951 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:47.345070 28951 webserver.cc:460] Webserver started at http://127.3.147.62:32991/ using document root <none> and password file <none>
I20260812 09:44:47.345618 28951 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:47.345721 28951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:47.345942 28951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:47.347930 28951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "e377a07dd3fd421e921b8e99102903ef"
format_stamp: "Formatted at 2026-08-12 09:44:47 on dist-test-slave-cbbz"
server_key: "e96a84f660712b9886fcf262839a1b7e"
server_key_iv: "cd42da524c6f68fa2132ccd0a26b52fd"
server_key_version: "encryptionkey@0"
I20260812 09:44:47.348539 28951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "e377a07dd3fd421e921b8e99102903ef"
format_stamp: "Formatted at 2026-08-12 09:44:47 on dist-test-slave-cbbz"
server_key: "e96a84f660712b9886fcf262839a1b7e"
server_key_iv: "cd42da524c6f68fa2132ccd0a26b52fd"
server_key_version: "encryptionkey@0"
I20260812 09:44:47.352270 28951 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:47.354534 28970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:47.355698 28951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:47.355841 28951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "e377a07dd3fd421e921b8e99102903ef"
format_stamp: "Formatted at 2026-08-12 09:44:47 on dist-test-slave-cbbz"
server_key: "e96a84f660712b9886fcf262839a1b7e"
server_key_iv: "cd42da524c6f68fa2132ccd0a26b52fd"
server_key_version: "encryptionkey@0"
I20260812 09:44:47.355947 28951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:47.373302 28951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:47.376041 28951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:47.376277 28951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:47.383405 28951 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:46527
I20260812 09:44:47.383432 29033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:46527 every 8 connection(s)
I20260812 09:44:47.384454 28951 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:47.387077 29034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:47.392561 29034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef: Bootstrap starting.
I20260812 09:44:47.393842 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 28951
I20260812 09:44:47.393976 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:47.394299 3660 external_mini_cluster.cc:1428] Setting key c340aedc4a5b01b2acd6d848a9b03154
I20260812 09:44:47.395427 29034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:47.396414 29034 log.cc:824] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:47.398741 29034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef: No bootstrap required, opened a new log
I20260812 09:44:47.402880 29034 raft_consensus.cc:348] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e377a07dd3fd421e921b8e99102903ef" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46527 } }
I20260812 09:44:47.403098 29034 raft_consensus.cc:374] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:47.403177 29034 raft_consensus.cc:729] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e377a07dd3fd421e921b8e99102903ef, State: Initialized, Role: FOLLOWER
I20260812 09:44:47.403654 29034 consensus_queue.cc:260] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e377a07dd3fd421e921b8e99102903ef" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46527 } }
I20260812 09:44:47.403824 29034 raft_consensus.cc:388] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 12 09:44:47 dist-test-slave-cbbz krb5kdc[28936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:47.403919 29034 raft_consensus.cc:482] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:47.404021 29034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:47.405047 29034 raft_consensus.cc:504] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e377a07dd3fd421e921b8e99102903ef" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46527 } }
I20260812 09:44:47.405385 29034 leader_election.cc:302] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e377a07dd3fd421e921b8e99102903ef; no voters:
I20260812 09:44:47.405686 29034 leader_election.cc:288] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:47.405798 29040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:47.406011 29040 raft_consensus.cc:686] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [term 1 LEADER]: Becoming Leader. State: Replica: e377a07dd3fd421e921b8e99102903ef, State: Running, Role: LEADER
I20260812 09:44:47.406387 29040 consensus_queue.cc:237] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e377a07dd3fd421e921b8e99102903ef" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46527 } }
I20260812 09:44:47.406996 29034 sys_catalog.cc:564] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:47.408092 29041 sys_catalog.cc:455] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e377a07dd3fd421e921b8e99102903ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e377a07dd3fd421e921b8e99102903ef" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46527 } } }
I20260812 09:44:47.408244 29041 sys_catalog.cc:458] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:47.408535 29042 sys_catalog.cc:455] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader e377a07dd3fd421e921b8e99102903ef. Latest consensus state: current_term: 1 leader_uuid: "e377a07dd3fd421e921b8e99102903ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e377a07dd3fd421e921b8e99102903ef" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 46527 } } }
I20260812 09:44:47.408682 29042 sys_catalog.cc:458] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:47.409396 29052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:47.410648 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:47.395426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50823 (local address 127.3.147.62:46527)
0812 09:44:47.395897 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:47.395913 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:47.395948 (+ 35us) server_negotiation.cc:407] Connection header received
0812 09:44:47.397001 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:47.397025 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:47.397452 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:47.397827 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:47.398541 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:47.399996 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:47.400876 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:47.401684 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:47.401896 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:47.404587 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:47.404592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:47.404604 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:47.404629 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:47.406716 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:47.407248 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:47.407254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:47.407259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:47.407407 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:47.407782 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:47.407787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:47.407789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:47.408386 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:47.408601 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:47.409413 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 09:44:47.409706 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":149,"threads_started":1}
I20260812 09:44:47.412504 29052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:47.413643 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:47.418401 29052 catalog_manager.cc:1347] Generated new cluster ID: e6b3cd30006b4e97a6dd463708dec658
I20260812 09:44:47.418457 29052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:47.426826 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:47.415905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57497 (local address 127.3.147.62:46527)
0812 09:44:47.416002 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:47.416008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:47.416274 (+ 266us) server_negotiation.cc:407] Connection header received
0812 09:44:47.416560 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:47.416562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:47.416600 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:47.416692 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:47.417521 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:47.418239 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:47.421323 (+ 3084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:47.421712 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:47.421844 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:47.423369 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:47.423373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:47.423376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:47.423402 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:47.425164 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:47.425625 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:47.425629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:47.425631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:47.425719 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:47.425984 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:47.425987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:47.425989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:47.426209 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:47.426340 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:47.426554 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 09:44:47.426669 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 09:44:47.433516 29052 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:47.434423 29052 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:47.443207 29052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e377a07dd3fd421e921b8e99102903ef: Generated new TSK 0
I20260812 09:44:47.444120 29052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:47.453244 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 28951
2026-08-12T09:44:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:47 dist-test-slave-cbbz krb5kdc[29071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:47 dist-test-slave-cbbz krb5kdc[29071](info): set up 2 sockets
Aug 12 09:44:47 dist-test-slave-cbbz krb5kdc[29071](info): commencing operation
krb5kdc: starting...
W20260812 09:44:48.561033 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:48 dist-test-slave-cbbz krb5kdc[29071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:48.688699 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:37979
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:37979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:48.800436 29085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:48.800706 29085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:48.800779 29085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:48.804783 29085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:48.804863 29085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:48.804901 29085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:48.804957 29085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:48.805002 29085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:48.810117 29085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:37979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:37979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:48.811112 29085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:48.812172 29085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:48.820648 29092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:48.820612 29094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:48.820691 29085 server_base.cc:1034] running on GCE node
W20260812 09:44:48.820885 29090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:48.821465 29085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:48.822484 29085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:48.823674 29085 hybrid_clock.cc:648] HybridClock initialized: now 1786527888823663 us; error 32 us; skew 500 ppm
Aug 12 09:44:48 dist-test-slave-cbbz krb5kdc[29071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527888, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:48.826404 29085 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:48.827309 29085 webserver.cc:460] Webserver started at http://127.3.147.62:33693/ using document root <none> and password file <none>
I20260812 09:44:48.827867 29085 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:48.827960 29085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:48.828253 29085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:48.830282 29085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "cc8cddbdc181434bad861b718ef60bbe"
format_stamp: "Formatted at 2026-08-12 09:44:48 on dist-test-slave-cbbz"
server_key: "b569ffd5a15f336032e731be5d871110"
server_key_iv: "04397cd3c2ac096905385b52c16926e9"
server_key_version: "encryptionkey@0"
I20260812 09:44:48.830849 29085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "cc8cddbdc181434bad861b718ef60bbe"
format_stamp: "Formatted at 2026-08-12 09:44:48 on dist-test-slave-cbbz"
server_key: "b569ffd5a15f336032e731be5d871110"
server_key_iv: "04397cd3c2ac096905385b52c16926e9"
server_key_version: "encryptionkey@0"
I20260812 09:44:48.834669 29085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260812 09:44:48.837082 29103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:48.838330 29085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:48.838488 29085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "cc8cddbdc181434bad861b718ef60bbe"
format_stamp: "Formatted at 2026-08-12 09:44:48 on dist-test-slave-cbbz"
server_key: "b569ffd5a15f336032e731be5d871110"
server_key_iv: "04397cd3c2ac096905385b52c16926e9"
server_key_version: "encryptionkey@0"
I20260812 09:44:48.838613 29085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:48.853542 29085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:48.856544 29085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:48.856757 29085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:48.864259 29085 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:37979
I20260812 09:44:48.864288 29167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:37979 every 8 connection(s)
I20260812 09:44:48.865228 29085 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:48.867960 29169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:48.873531 29169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe: Bootstrap starting.
I20260812 09:44:48.875017 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 29085
I20260812 09:44:48.875151 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:48.875464 3660 external_mini_cluster.cc:1428] Setting key 9f43d5ff8b75194a18cd1b9477ad3b3a
I20260812 09:44:48.876432 29169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:48.877332 29169 log.cc:824] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:48.879848 29169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe: No bootstrap required, opened a new log
I20260812 09:44:48.883404 29169 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc8cddbdc181434bad861b718ef60bbe" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37979 } }
I20260812 09:44:48.883658 29169 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:48.883757 29169 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc8cddbdc181434bad861b718ef60bbe, State: Initialized, Role: FOLLOWER
I20260812 09:44:48.884290 29169 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc8cddbdc181434bad861b718ef60bbe" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37979 } }
I20260812 09:44:48.884423 29169 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:48.884514 29169 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:48.884625 29169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:48.884889 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:48.876631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55991 (local address 127.3.147.62:37979)
0812 09:44:48.877111 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:48.877143 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:48.877181 (+ 38us) server_negotiation.cc:407] Connection header received
0812 09:44:48.878085 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:48.878114 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:48.878495 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:48.878840 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:48.879300 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:48.880404 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:48.881324 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:48.881817 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:48.881994 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:48.882881 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:48.882886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:48.882899 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:48.883300 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:48.883353 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:48.883376 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:48.883485 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:48.883656 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 09:44:48.883897 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":144,"threads_started":1}
I20260812 09:44:48.885612 29169 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc8cddbdc181434bad861b718ef60bbe" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37979 } }
I20260812 09:44:48.886001 29169 leader_election.cc:302] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc8cddbdc181434bad861b718ef60bbe; no voters:
I20260812 09:44:48.886336 29169 leader_election.cc:288] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:48.886443 29174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:48.886693 29174 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [term 1 LEADER]: Becoming Leader. State: Replica: cc8cddbdc181434bad861b718ef60bbe, State: Running, Role: LEADER
I20260812 09:44:48.887013 29174 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc8cddbdc181434bad861b718ef60bbe" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37979 } }
I20260812 09:44:48.887692 29169 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:48.888092 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:48.889020 29176 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc8cddbdc181434bad861b718ef60bbe. Latest consensus state: current_term: 1 leader_uuid: "cc8cddbdc181434bad861b718ef60bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc8cddbdc181434bad861b718ef60bbe" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37979 } } }
I20260812 09:44:48.889029 29175 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc8cddbdc181434bad861b718ef60bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc8cddbdc181434bad861b718ef60bbe" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 37979 } } }
I20260812 09:44:48.889175 29175 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:48.889160 29176 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe [sys.catalog]: This master's current role is: LEADER
W20260812 09:44:48.893337 29195 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:48.893432 29195 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:48.893543 29185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:48.894485 29185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:48.895613 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:48.890715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54537 (local address 127.3.147.62:37979)
0812 09:44:48.891004 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:48.891009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:48.891248 (+ 239us) server_negotiation.cc:407] Connection header received
0812 09:44:48.891442 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:48.891445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:48.891492 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:48.891605 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:48.891938 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:48.892732 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:48.893846 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:48.894182 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:48.894293 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:48.895171 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:48.895175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:48.895178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:48.895190 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:48.895218 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:48.895231 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:48.895310 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:48.895435 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 09:44:48.895487 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260812 09:44:48.900363 29185 catalog_manager.cc:1347] Generated new cluster ID: a1d8d818da4f4785a71a3d87153179b1
I20260812 09:44:48.900451 29185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:48.911032 29185 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:48.912380 29185 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:48.922133 29185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc8cddbdc181434bad861b718ef60bbe: Generated new TSK 0
I20260812 09:44:48.923022 29185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:48.933243 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 29085
2026-08-12T09:44:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:48 dist-test-slave-cbbz krb5kdc[29203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:48 dist-test-slave-cbbz krb5kdc[29203](info): set up 2 sockets
Aug 12 09:44:48 dist-test-slave-cbbz krb5kdc[29203](info): commencing operation
krb5kdc: starting...
W20260812 09:44:50.053102 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:50 dist-test-slave-cbbz krb5kdc[29203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:50.178954 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43709
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:40937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:43709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:50.291342 29220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:50.291589 29220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:50.291657 29220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:50.295627 29220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:50.295686 29220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:50.295704 29220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:50.295765 29220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:50.295804 29220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:50.300947 29220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:40937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:43709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:43709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:50.301985 29220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:50.302973 29220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:50.311369 29230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:50.311373 29225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:50.311373 29227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:50.311470 29220 server_base.cc:1034] running on GCE node
I20260812 09:44:50.312243 29220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:50.313257 29220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:50.314414 29220 hybrid_clock.cc:648] HybridClock initialized: now 1786527890314392 us; error 34 us; skew 500 ppm
Aug 12 09:44:50 dist-test-slave-cbbz krb5kdc[29203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:50.317222 29220 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:50.318084 29220 webserver.cc:460] Webserver started at http://127.3.147.62:34587/ using document root <none> and password file <none>
I20260812 09:44:50.318621 29220 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:50.318686 29220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:50.318924 29220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:50.320989 29220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "70927a9ff48b4db8b185ef2b43ce30ec"
format_stamp: "Formatted at 2026-08-12 09:44:50 on dist-test-slave-cbbz"
server_key: "e22d55833eb2132e25457c0c51f0d5a3"
server_key_iv: "a35ce7e453a8beb89172119b2f2a94f8"
server_key_version: "encryptionkey@0"
I20260812 09:44:50.321590 29220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "70927a9ff48b4db8b185ef2b43ce30ec"
format_stamp: "Formatted at 2026-08-12 09:44:50 on dist-test-slave-cbbz"
server_key: "e22d55833eb2132e25457c0c51f0d5a3"
server_key_iv: "a35ce7e453a8beb89172119b2f2a94f8"
server_key_version: "encryptionkey@0"
I20260812 09:44:50.325322 29220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:50.327586 29239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:50.328792 29220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:50.328931 29220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "70927a9ff48b4db8b185ef2b43ce30ec"
format_stamp: "Formatted at 2026-08-12 09:44:50 on dist-test-slave-cbbz"
server_key: "e22d55833eb2132e25457c0c51f0d5a3"
server_key_iv: "a35ce7e453a8beb89172119b2f2a94f8"
server_key_version: "encryptionkey@0"
I20260812 09:44:50.329039 29220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:50.333446 29220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:50.336648 29220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:50.336839 29220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:50.344033 29220 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:43709
I20260812 09:44:50.344062 29301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:43709 every 8 connection(s)
I20260812 09:44:50.344950 29220 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:50.345268 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 29220
I20260812 09:44:50.345422 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:50.345745 3660 external_mini_cluster.cc:1428] Setting key c8077fa9149839040f6f56267bdaff89
I20260812 09:44:50.348100 29302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:50.351325 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:50.347153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34137 (local address 127.3.147.62:43709)
0812 09:44:50.347601 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:50.347617 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:50.347651 (+ 34us) server_negotiation.cc:407] Connection header received
0812 09:44:50.348374 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:50.348392 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:50.348735 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:50.349121 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:50.350137 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:50.350142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:50.350163 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:50.350179 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:50.350233 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:50.350252 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:50.350403 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:50.350474 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 09:44:50.350622 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":144,"threads_started":1}
I20260812 09:44:50.354007 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:50.354475 29302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec: Bootstrap starting.
I20260812 09:44:50.357296 29302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:50.357847 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:50.356097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60305 (local address 127.3.147.62:43709)
0812 09:44:50.356259 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:50.356263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:50.356510 (+ 247us) server_negotiation.cc:407] Connection header received
0812 09:44:50.356628 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:50.356630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:50.356657 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:50.356730 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:50.357436 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:50.357439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:50.357441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 09:44:50.357451 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:50.357474 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 09:44:50.357483 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:50.357552 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:50.357683 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 09:44:50.357711 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 09:44:50.358253 29302 log.cc:824] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:50.360057 29302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec: No bootstrap required, opened a new log
I20260812 09:44:50.363438 29302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70927a9ff48b4db8b185ef2b43ce30ec" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43709 } }
I20260812 09:44:50.363678 29302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 09:44:50.363761 29302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70927a9ff48b4db8b185ef2b43ce30ec, State: Initialized, Role: FOLLOWER
I20260812 09:44:50.364209 29302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70927a9ff48b4db8b185ef2b43ce30ec" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43709 } }
I20260812 09:44:50.364363 29302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 09:44:50.364428 29302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 09:44:50.364524 29302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:50.365353 29302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70927a9ff48b4db8b185ef2b43ce30ec" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43709 } }
I20260812 09:44:50.365687 29302 leader_election.cc:302] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70927a9ff48b4db8b185ef2b43ce30ec; no voters:
I20260812 09:44:50.365984 29302 leader_election.cc:288] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 09:44:50.366093 29316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:50.366346 29316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [term 1 LEADER]: Becoming Leader. State: Replica: 70927a9ff48b4db8b185ef2b43ce30ec, State: Running, Role: LEADER
I20260812 09:44:50.366672 29316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70927a9ff48b4db8b185ef2b43ce30ec" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43709 } }
I20260812 09:44:50.367195 29302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:50.368356 29318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70927a9ff48b4db8b185ef2b43ce30ec. Latest consensus state: current_term: 1 leader_uuid: "70927a9ff48b4db8b185ef2b43ce30ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70927a9ff48b4db8b185ef2b43ce30ec" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43709 } } }
I20260812 09:44:50.368497 29318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:50.368891 29325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:50.368854 29317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70927a9ff48b4db8b185ef2b43ce30ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70927a9ff48b4db8b185ef2b43ce30ec" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 43709 } } }
I20260812 09:44:50.369005 29317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:50.371737 29325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:50.377590 29325 catalog_manager.cc:1347] Generated new cluster ID: cc36c4e6a42646d280f394519328d9c9
I20260812 09:44:50.378152 29325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:50.412806 29325 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:50.413131 29325 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:50.419261 29325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70927a9ff48b4db8b185ef2b43ce30ec: Generated new TSK 0
I20260812 09:44:50.419895 29325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 09:44:50.425884 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 29220
2026-08-12T09:44:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1493 ms)
[----------] 18 tests from AuthTokenIssuingTest (26961 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:50 dist-test-slave-cbbz krb5kdc[29339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:50 dist-test-slave-cbbz krb5kdc[29339](info): set up 2 sockets
Aug 12 09:44:50 dist-test-slave-cbbz krb5kdc[29339](info): commencing operation
krb5kdc: starting...
W20260812 09:44:51.549274 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:51 dist-test-slave-cbbz krb5kdc[29339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:51.675450 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:45719
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:45719,127.3.147.61:34697,127.3.147.60:35123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:51.786970 29353 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 09:44:51.787261 29353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:51.787284 29353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:51.787320 29353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:51.791198 29353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:51.791252 29353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:51.791271 29353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:51.791287 29353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:51.791302 29353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:51.796396 29353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:45719,127.3.147.61:34697,127.3.147.60:35123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:45719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:51.797323 29353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:51.798285 29353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:51.806819 29363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:51.806847 29361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:51.806870 29365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:51.807030 29353 server_base.cc:1034] running on GCE node
I20260812 09:44:51.807711 29353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:51.808851 29353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:51.810022 29353 hybrid_clock.cc:648] HybridClock initialized: now 1786527891809998 us; error 40 us; skew 500 ppm
Aug 12 09:44:51 dist-test-slave-cbbz krb5kdc[29339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527891, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:51.817975 29353 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:51.818935 29353 webserver.cc:460] Webserver started at http://127.3.147.62:43205/ using document root <none> and password file <none>
I20260812 09:44:51.819469 29353 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:51.819531 29353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:51.819707 29353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:51.821776 29353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "25129173de6746d79239306724474817"
format_stamp: "Formatted at 2026-08-12 09:44:51 on dist-test-slave-cbbz"
server_key: "43ea556812c69f111895969468e96562"
server_key_iv: "16ee5703970c2719bc10416f4023cc81"
server_key_version: "encryptionkey@0"
I20260812 09:44:51.822321 29353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "25129173de6746d79239306724474817"
format_stamp: "Formatted at 2026-08-12 09:44:51 on dist-test-slave-cbbz"
server_key: "43ea556812c69f111895969468e96562"
server_key_iv: "16ee5703970c2719bc10416f4023cc81"
server_key_version: "encryptionkey@0"
I20260812 09:44:51.826175 29353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 09:44:51.828630 29374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:51.829883 29353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 09:44:51.830027 29353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "25129173de6746d79239306724474817"
format_stamp: "Formatted at 2026-08-12 09:44:51 on dist-test-slave-cbbz"
server_key: "43ea556812c69f111895969468e96562"
server_key_iv: "16ee5703970c2719bc10416f4023cc81"
server_key_version: "encryptionkey@0"
I20260812 09:44:51.830142 29353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:51.849195 29353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:51.851962 29353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:51.852222 29353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:51.859118 29353 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:45719
I20260812 09:44:51.859414 29434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:45719 every 8 connection(s)
I20260812 09:44:51.860100 29353 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:51.862105 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 29353
I20260812 09:44:51.862263 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:51.862538 3660 external_mini_cluster.cc:1428] Setting key 69c07f4238ecb53b32bfbcbe42c34f48
I20260812 09:44:51.863065 29435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:51.866875 29435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.147.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.61@KRBTEST.COM" created.
Aug 12 09:44:51 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527891, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:51.883234 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:51.869272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:45719 (local address 127.0.0.1:41648)
0812 09:44:51.869846 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:51.869883 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:51.870709 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:51.871389 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:51.871394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:51.871893 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:51.872297 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:51.872312 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:51.873693 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:51.873696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:51.874459 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:51.874469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:51.875124 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:51.875128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:51.875250 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:51.875872 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:51.875888 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:51.877665 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:51.879873 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:51.879884 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:51.879895 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:51.880206 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:51.880575 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:51.880580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:51.880582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:51.880719 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:51.881363 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:51.881370 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:51.881548 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 09:44:51.881733 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 09:44:51.882167 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":120,"thread_start_us":164,"threads_started":1}
I20260812 09:44:51.883236 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:51.869272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.3.147.62:45719)
0812 09:44:51.869704 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:51.869723 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:51.869994 (+ 271us) server_negotiation.cc:407] Connection header received
0812 09:44:51.871015 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:51.871020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:51.871197 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:51.871389 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:51.872453 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:51.873549 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:51.874584 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:51.874958 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:51.875131 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:51.877825 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:51.877830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:51.877842 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:51.877863 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:51.879719 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:51.880339 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:51.880345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:51.880347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:51.880467 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:51.880826 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:51.880829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:51.880831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:51.881237 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:51.881427 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:51.881831 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 09:44:51.882112 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":268,"thread_start_us":130,"threads_started":1}
I20260812 09:44:51.885876 29435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:51.886673 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:51.886387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:50982)
0812 09:44:51.886513 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:51.886608 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 09:44:51.886888 29375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
I20260812 09:44:51.887797 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:51.887567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:50984)
0812 09:44:51.887684 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:51.887732 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 09:44:51.887946 29435 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:34697: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab.
Entry for principal kudu/127.3.147.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab.
I20260812 09:44:51.891702 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.61:34697
--webserver_interface=127.3.147.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab
--principal=kudu/127.3.147.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:45719,127.3.147.61:34697,127.3.147.60:35123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:44:51.937202 29435 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } attempt: 1
I20260812 09:44:51.938176 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:51.937790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:50992)
0812 09:44:51.937989 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:51.938075 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 09:44:51.939011 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:51.938770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:51004)
0812 09:44:51.938893 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:51.938952 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 09:44:51.939205 29435 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:34697: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
W20260812 09:44:52.006299 29445 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 09:44:52.006651 29445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:52.006726 29445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:52.006806 29445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 09:44:52.007359 29435 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } attempt: 2
I20260812 09:44:52.008294 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.007935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:51016)
0812 09:44:52.008091 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.008210 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 09:44:52.008980 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.008783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:51026)
0812 09:44:52.008880 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.008926 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 09:44:52.009136 29435 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:34697: Network error: Client connection negotiation failed: client connection to 127.3.147.61:34697: connect: Connection refused (error 111)
W20260812 09:44:52.011529 29445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:52.011581 29445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:52.011637 29445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:52.011672 29445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:52.011696 29445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:52.016947 29445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:45719,127.3.147.61:34697,127.3.147.60:35123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab
--principal=kudu/127.3.147.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.61:34697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.147.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:52.018045 29445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:52.019039 29445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:52.027539 29452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:52.027655 29450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:52.027729 29445 server_base.cc:1034] running on GCE node
W20260812 09:44:52.027539 29454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:52.028529 29445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:52.029608 29445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:52.030823 29445 hybrid_clock.cc:648] HybridClock initialized: now 1786527892030804 us; error 36 us; skew 500 ppm
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527892, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:52.033692 29445 init.cc:377] Logged in from keytab as kudu/127.3.147.61@KRBTEST.COM (short username kudu)
I20260812 09:44:52.034536 29445 webserver.cc:460] Webserver started at http://127.3.147.61:40157/ using document root <none> and password file <none>
I20260812 09:44:52.035045 29445 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:52.035102 29445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:52.035264 29445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:52.037283 29445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/data/instance:
uuid: "9b81a4f07dbc4feebe61533d6faa3255"
format_stamp: "Formatted at 2026-08-12 09:44:52 on dist-test-slave-cbbz"
server_key: "da6ec6149bc800c2277afe22fc0ff319"
server_key_iv: "c0a61d073f6d4406e83e481c4d422492"
server_key_version: "encryptionkey@0"
I20260812 09:44:52.037797 29445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/wal/instance:
uuid: "9b81a4f07dbc4feebe61533d6faa3255"
format_stamp: "Formatted at 2026-08-12 09:44:52 on dist-test-slave-cbbz"
server_key: "da6ec6149bc800c2277afe22fc0ff319"
server_key_iv: "c0a61d073f6d4406e83e481c4d422492"
server_key_version: "encryptionkey@0"
I20260812 09:44:52.041410 29445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 09:44:52.043704 29465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:52.044997 29445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 09:44:52.045109 29445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/wal
uuid: "9b81a4f07dbc4feebe61533d6faa3255"
format_stamp: "Formatted at 2026-08-12 09:44:52 on dist-test-slave-cbbz"
server_key: "da6ec6149bc800c2277afe22fc0ff319"
server_key_iv: "c0a61d073f6d4406e83e481c4d422492"
server_key_version: "encryptionkey@0"
I20260812 09:44:52.045190 29445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:52.061367 29445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:52.064636 29445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:52.064800 29445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:52.071943 29445 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.61:34697
I20260812 09:44:52.072283 29524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.61:34697 every 8 connection(s)
I20260812 09:44:52.072983 29445 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
I20260812 09:44:52.075727 29525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:52.078778 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 29445
I20260812 09:44:52.078915 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-1/wal/instance
I20260812 09:44:52.079253 3660 external_mini_cluster.cc:1428] Setting key f044ec3eb1e22ae80d50d408d625d933
I20260812 09:44:52.079826 29525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.147.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.60@KRBTEST.COM" created.
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527892, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:52.096428 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.082193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.3.147.62:45719)
0812 09:44:52.082329 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.082332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.083365 (+ 1033us) server_negotiation.cc:407] Connection header received
0812 09:44:52.084373 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.084377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.084448 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.084541 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.085689 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.086222 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.087397 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.087634 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.087745 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.091765 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.091768 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.091770 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.091795 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.094226 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.094733 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.094738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.094740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.094824 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.095193 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.095196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.095198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.095399 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.095520 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.096084 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.096229 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 09:44:52.097244 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.082692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:45719 (local address 127.0.0.1:41660)
0812 09:44:52.083208 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.083250 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:52.084093 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:52.084717 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:52.084724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:52.085062 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:52.085535 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.085547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.086390 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.086393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.087249 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.087258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.087787 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.087791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.088003 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.088619 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:52.088638 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:52.091587 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:52.094346 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.094352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.094365 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.094626 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.094918 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.094922 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.094924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.095064 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.095522 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:52.095529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:52.095801 (+ 272us) client_negotiation.cc:769] Sending connection context
0812 09:44:52.096031 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 09:44:52.096397 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":150,"threads_started":1}
I20260812 09:44:52.098897 29525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:52.100302 29435 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } attempt: 3
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527892, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527891, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab.
Entry for principal kudu/127.3.147.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab.
I20260812 09:44:52.108213 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.60:35123
--webserver_interface=127.3.147.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab
--principal=kudu/127.3.147.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:45719,127.3.147.61:34697,127.3.147.60:35123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:44:52.110837 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.100716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:51054)
0812 09:44:52.100887 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.100904 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:52.100981 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:52.101326 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:52.101328 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:52.101489 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:52.101637 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.101648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.102693 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.102697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.103363 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.103374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.103909 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.103913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.104029 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.104517 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:52.104529 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:52.106109 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:52.108818 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.108822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.108825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.109118 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.109559 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.109566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.109571 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.109695 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.110249 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:52.110252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:52.110362 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 09:44:52.110496 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 09:44:52.110653 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 09:44:52.110826 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.099458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:51038)
0812 09:44:52.099851 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.099863 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:52.099929 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:52.100372 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:52.100376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:52.100546 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:52.100647 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.100652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.102099 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.102101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.102723 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.102736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.103362 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.103366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.103481 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.104008 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:52.104029 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:52.105577 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:52.108893 (+ 3316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.108896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.108898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.109157 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.109581 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.109586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.109591 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.109707 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.110281 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:52.110286 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:52.110374 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 09:44:52.110455 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 09:44:52.110599 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":1}
I20260812 09:44:52.110980 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.100783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51054 (local address 127.3.147.61:34697)
0812 09:44:52.101097 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.101116 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.101141 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:44:52.101173 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.101176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.101213 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.101299 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.101783 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.102573 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.103485 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.103792 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.103903 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.106312 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.106314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.106316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.106337 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.108695 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.109256 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.109265 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.109270 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.109417 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.109829 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.109834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.109836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.110106 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.110288 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.110606 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.110760 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"spinlock_wait_cycles":7040,"thread_start_us":110,"threads_started":1}
I20260812 09:44:52.111467 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.099560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.3.147.61:34697)
0812 09:44:52.099874 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.099884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.099904 (+ 20us) server_negotiation.cc:407] Connection header received
0812 09:44:52.100013 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.100017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.100218 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.100417 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.100767 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.101977 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.102846 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.103248 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.103328 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.105799 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.105813 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.105830 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.105860 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.108787 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.109269 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.109272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.109274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.109447 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.109833 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.109839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.109842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.110165 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.111028 (+ 863us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.111111 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.111253 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"spinlock_wait_cycles":6784,"thread_start_us":135,"threads_started":1}
I20260812 09:44:52.112890 29525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:52.113310 29435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:52.113657 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.113342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50312)
0812 09:44:52.113489 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.113573 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 09:44:52.114087 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.113808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50326)
0812 09:44:52.113930 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.114008 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 09:44:52.114051 29467 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
I20260812 09:44:52.114944 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.114737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50332)
0812 09:44:52.114851 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.114896 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 09:44:52.115247 29525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:35123: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
I20260812 09:44:52.115541 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.115261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50338)
0812 09:44:52.115416 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.115470 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260812 09:44:52.115728 29435 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:35123: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
I20260812 09:44:52.146894 29435 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } attempt: 1
I20260812 09:44:52.147773 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.147411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50340)
0812 09:44:52.147588 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.147663 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 09:44:52.148571 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.148333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50344)
0812 09:44:52.148450 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.148508 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 09:44:52.148785 29435 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:35123: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
I20260812 09:44:52.164522 29525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } attempt: 1
I20260812 09:44:52.165382 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.165069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50358)
0812 09:44:52.165239 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.165303 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 09:44:52.166137 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.165939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50372)
0812 09:44:52.166041 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.166085 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 09:44:52.166342 29525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:35123: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
I20260812 09:44:52.223989 29435 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } attempt: 2
I20260812 09:44:52.224912 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.224570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50380)
0812 09:44:52.224744 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.224814 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 09:44:52.225020 29533 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 09:44:52.225446 29533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:52.225499 29533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:52.225567 29533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 09:44:52.225757 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.225508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50394)
0812 09:44:52.225634 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.225689 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 09:44:52.225977 29435 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:35123: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
W20260812 09:44:52.229977 29533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:52.230033 29533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:52.230053 29533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:52.230070 29533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:52.230085 29533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:52.234509 29525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } attempt: 2
I20260812 09:44:52.235360 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.235070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50408)
0812 09:44:52.235221 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.235283 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 09:44:52.235888 29533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:45719,127.3.147.61:34697,127.3.147.60:35123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab
--principal=kudu/127.3.147.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.60:35123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.147.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:52.236191 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.235872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50422)
0812 09:44:52.235988 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.236069 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 09:44:52.236334 29525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:35123: Network error: Client connection negotiation failed: client connection to 127.3.147.60:35123: connect: Connection refused (error 111)
I20260812 09:44:52.237162 29533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:52.238231 29533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:52.247027 29540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:52.247054 29539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:52.247112 29533 server_base.cc:1034] running on GCE node
W20260812 09:44:52.247291 29542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:52.247924 29533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:52.249262 29533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:52.250458 29533 hybrid_clock.cc:648] HybridClock initialized: now 1786527892250407 us; error 61 us; skew 500 ppm
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527892, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:52.253295 29533 init.cc:377] Logged in from keytab as kudu/127.3.147.60@KRBTEST.COM (short username kudu)
I20260812 09:44:52.254199 29533 webserver.cc:460] Webserver started at http://127.3.147.60:44273/ using document root <none> and password file <none>
I20260812 09:44:52.254729 29533 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:52.254789 29533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:52.254952 29533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:52.257118 29533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/data/instance:
uuid: "dc058334f1964977b4513980e01bb295"
format_stamp: "Formatted at 2026-08-12 09:44:52 on dist-test-slave-cbbz"
server_key: "e1a0f50e7367c274b34283c0022df9cc"
server_key_iv: "5a9da112622d626a62f2dff2745e4ac1"
server_key_version: "encryptionkey@0"
I20260812 09:44:52.257640 29533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/wal/instance:
uuid: "dc058334f1964977b4513980e01bb295"
format_stamp: "Formatted at 2026-08-12 09:44:52 on dist-test-slave-cbbz"
server_key: "e1a0f50e7367c274b34283c0022df9cc"
server_key_iv: "5a9da112622d626a62f2dff2745e4ac1"
server_key_version: "encryptionkey@0"
I20260812 09:44:52.261283 29533 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 09:44:52.263576 29549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:52.264829 29533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:44:52.264968 29533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/wal
uuid: "dc058334f1964977b4513980e01bb295"
format_stamp: "Formatted at 2026-08-12 09:44:52 on dist-test-slave-cbbz"
server_key: "e1a0f50e7367c274b34283c0022df9cc"
server_key_iv: "5a9da112622d626a62f2dff2745e4ac1"
server_key_version: "encryptionkey@0"
I20260812 09:44:52.265079 29533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:52.273859 29533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:52.276675 29533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:52.276860 29533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:52.283705 29533 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.60:35123
I20260812 09:44:52.284017 29610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.60:35123 every 8 connection(s)
I20260812 09:44:52.284726 29533 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
I20260812 09:44:52.286940 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 29533
I20260812 09:44:52.287041 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786527603645036-3660-0/minicluster-data/master-2/wal/instance
I20260812 09:44:52.287333 3660 external_mini_cluster.cc:1428] Setting key cb8adf24594de85e9968a9ea2807d3e6
I20260812 09:44:52.287539 29613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:52.292783 29613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527892, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:52.303107 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.288752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50438 (local address 127.3.147.60:35123)
0812 09:44:52.289166 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.289196 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.289240 (+ 44us) server_negotiation.cc:407] Connection header received
0812 09:44:52.289983 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.290006 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.290423 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.290798 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.291484 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.292963 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.293850 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.294481 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.294653 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.297717 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.297722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.297730 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.297755 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.299777 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.300293 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.300299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.300305 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.300425 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.300748 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.300751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.300753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.301273 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.301484 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.301798 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.302091 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":111,"threads_started":1}
I20260812 09:44:52.304023 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.294646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:45719 (local address 127.0.0.1:41664)
0812 09:44:52.295036 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.295075 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:52.295221 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:52.295563 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:52.295572 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:52.296072 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:52.296302 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.296314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.297341 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.297344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.297944 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.297977 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.298432 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.298437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.298578 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.298991 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:52.299005 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:52.300584 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:52.302499 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.302508 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.302511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.302730 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.303026 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.303028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.303030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.303102 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.303558 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:52.303563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:52.303659 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 09:44:52.303770 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 09:44:52.303891 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":78,"threads_started":1}
I20260812 09:44:52.304833 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.294707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41664 (local address 127.3.147.62:45719)
0812 09:44:52.294834 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.294838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.295166 (+ 328us) server_negotiation.cc:407] Connection header received
0812 09:44:52.295341 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.295344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.295402 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.295467 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.296431 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.297213 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.298095 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.298305 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.298384 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.300749 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.300752 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.300755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.300779 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.302394 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.302830 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.302834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.302835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.302928 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.303213 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.303216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.303218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.303453 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.303555 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.303852 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.303970 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 09:44:52.305892 29613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527892, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
I20260812 09:44:52.314273 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.305353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.3.147.62:45719)
0812 09:44:52.305444 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.305448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.305495 (+ 47us) server_negotiation.cc:407] Connection header received
0812 09:44:52.305597 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.305600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.305641 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.305715 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.306412 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.307136 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.307865 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.308085 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.308239 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.310686 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.310689 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.310691 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.310737 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.312548 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.312967 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.312973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.312978 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.313066 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.313389 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.313395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.313397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.313640 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.313877 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.313994 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.314117 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 09:44:52.314682 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.306395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:34697 (local address 127.0.0.1:51070)
0812 09:44:52.306495 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.306508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:52.306571 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:52.306803 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:52.306806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:52.306962 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:52.307064 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.307069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.307799 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.307804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.308392 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.308401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.308888 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.308892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.308986 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.309431 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:52.309442 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:52.311036 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:52.313055 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.313057 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.313058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.313329 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.313604 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.313606 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.313608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.313688 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.314163 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:52.314168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:52.314271 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 09:44:52.314401 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 09:44:52.314523 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 09:44:52.314718 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.306491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51070 (local address 127.3.147.61:34697)
0812 09:44:52.306606 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.306609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.306621 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:44:52.306658 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.306659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.306698 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.306763 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.307192 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.307686 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.308520 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.308769 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.308887 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.311197 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.311201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.311203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.311232 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.312949 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.313426 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.313432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.313435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.313510 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.313813 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.313817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.313819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.314043 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.314146 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.314463 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.314579 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 09:44:52.315505 29613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527892, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
I20260812 09:44:52.323603 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.316003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50450)
0812 09:44:52.316102 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.316114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:52.316211 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:52.316427 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:52.316430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:52.316585 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:52.316640 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.316644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.317338 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.317341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.317822 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.317828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.318288 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.318291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.318363 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.318827 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:52.318841 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:52.320161 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:52.322080 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.322085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.322088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.322318 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.322586 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.322589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.322591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.322696 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.323105 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:52.323108 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:52.323201 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 09:44:52.323308 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 09:44:52.323418 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 09:44:52.323635 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.316043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.3.147.60:35123)
0812 09:44:52.316217 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.316221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.316235 (+ 14us) server_negotiation.cc:407] Connection header received
0812 09:44:52.316291 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.316294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.316326 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.316416 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.316748 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.317225 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.317932 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.318181 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.318288 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.320310 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.320313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.320315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.320335 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.321970 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.322425 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.322428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.322429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.322499 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.322792 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.322798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.322802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.323008 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.323105 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.323392 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.323514 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260812 09:44:52.324570 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.315411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.3.147.61:34697)
0812 09:44:52.315513 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.315519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.315535 (+ 16us) server_negotiation.cc:407] Connection header received
0812 09:44:52.315680 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.315684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.315732 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.315794 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.316599 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.317085 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.317816 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.318042 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.318153 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.320977 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.320980 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.320983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.321006 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.322618 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.323105 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.323109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.323110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.323201 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.323527 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.323530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.323533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.323762 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.323964 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.324327 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.324447 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 09:44:52.325148 29435 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } attempt: 3
I20260812 09:44:52.327493 29525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } attempt: 3
I20260812 09:44:52.327888 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:52.328565 29613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295: Bootstrap starting.
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527891, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:44:52.329355 29404 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "25129173de6746d79239306724474817"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41666
I20260812 09:44:52.331489 29613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:52.332443 29613 log.cc:824] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295: Log is configured to *not* fsync() on all Append() calls
Aug 12 09:44:52 dist-test-slave-cbbz krb5kdc[29339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527892, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:44:52.334872 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.325528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50452)
0812 09:44:52.325634 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.325651 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:52.325739 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:52.326202 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:52.326205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:52.326382 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:52.326482 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.326488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.327321 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.327324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.328341 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.328353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.329151 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.329154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.329262 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.329729 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:52.329743 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:52.331141 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:52.333077 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.333080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.333082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.333353 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.333722 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.333725 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.333727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.333823 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.334303 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:52.334306 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:52.334431 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 09:44:52.334540 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 09:44:52.334709 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 09:44:52.335279 29613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295: No bootstrap required, opened a new log
I20260812 09:44:52.335460 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.325853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.3.147.60:35123)
0812 09:44:52.325962 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.325968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.325983 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:44:52.326014 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.326020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.326066 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.326154 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.326597 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.327200 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.328565 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.328857 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.329498 (+ 641us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.331285 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.331288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.331291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.331312 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.332960 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.333497 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.333500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.333502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.333606 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.333923 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.333926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.333928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.334142 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.335065 (+ 923us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.335142 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.335246 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:44:52.338194 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.327840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:35123 (local address 127.0.0.1:50462)
0812 09:44:52.327961 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:52.327975 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:52.328060 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:52.329022 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:52.329025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:52.329206 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:52.329326 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.329332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.330637 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.330641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.331374 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:52.331389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.332508 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.332512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:52.332603 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.333059 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:52.333073 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:52.334505 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:52.336720 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.336724 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.336726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.336917 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.337203 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:52.337206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:52.337208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:52.337289 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:52.337703 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:52.337708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:52.337797 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 09:44:52.337900 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 09:44:52.338029 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 09:44:52.338258 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.328506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.3.147.60:35123)
0812 09:44:52.328790 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.328799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.328812 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:44:52.328843 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.328846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.328883 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.329695 (+ 812us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.329789 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.330509 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.331780 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.332386 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.332489 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.334635 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.334639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.334641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.334665 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.336611 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.337023 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.337028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.337030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.337113 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.337383 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.337388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.337390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.337599 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.337689 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.337986 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.338091 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":118,"threads_started":1}
I20260812 09:44:52.338454 29435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817: Bootstrap starting.
I20260812 09:44:52.339409 29613 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } }
I20260812 09:44:52.339689 29613 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc058334f1964977b4513980e01bb295, State: Initialized, Role: FOLLOWER
I20260812 09:44:52.340037 29613 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } }
I20260812 09:44:52.340692 29621 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.340909 29621 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:52.341408 29613 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:52.341423 29435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:52.342339 29435 log.cc:824] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:52.342728 29525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255: Bootstrap starting.
I20260812 09:44:52.344607 29435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817: No bootstrap required, opened a new log
I20260812 09:44:52.345257 29525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:52.346063 29525 log.cc:824] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255: Log is configured to *not* fsync() on all Append() calls
W20260812 09:44:52.346691 29635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:52.346767 29635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:52.347769 29435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } }
I20260812 09:44:52.347988 29435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25129173de6746d79239306724474817, State: Initialized, Role: FOLLOWER
I20260812 09:44:52.348342 29435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } }
I20260812 09:44:52.348899 29639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.349193 29639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:52.349653 29435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:52.349749 29404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 09:44:52.349860 29404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:52.349954 29525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255: No bootstrap required, opened a new log
I20260812 09:44:52.351050 29404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } }
I20260812 09:44:52.351936 29404 leader_election.cc:288] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [CANDIDATE]: Term 1 election: Requested vote from peers 9b81a4f07dbc4feebe61533d6faa3255 (127.3.147.61:34697), dc058334f1964977b4513980e01bb295 (127.3.147.60:35123)
I20260812 09:44:52.353437 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "25129173de6746d79239306724474817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dc058334f1964977b4513980e01bb295"
I20260812 09:44:52.353688 29580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:52.353957 29525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } }
I20260812 09:44:52.354168 29525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b81a4f07dbc4feebe61533d6faa3255, State: Initialized, Role: FOLLOWER
I20260812 09:44:52.354508 29525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } }
I20260812 09:44:52.355055 29580 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25129173de6746d79239306724474817 in term 1.
W20260812 09:44:52.355585 29381 tablet.cc:2369] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 09:44:52.355625 29377 leader_election.cc:302] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25129173de6746d79239306724474817, dc058334f1964977b4513980e01bb295; no voters:
I20260812 09:44:52.355887 29525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:52.355933 29494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "25129173de6746d79239306724474817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9b81a4f07dbc4feebe61533d6faa3255"
I20260812 09:44:52.356165 29494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:52.357496 29494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25129173de6746d79239306724474817 in term 1.
W20260812 09:44:52.361260 29652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:52.361382 29652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:52.358974 29533 master_runner.cc:186] Error getting master registration for 127.3.147.61:34697: OK, instance_id { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" instance_seqno: 1786527892070047 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 09:44:52.362531 29533 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 09:44:52.359762 29353 master_runner.cc:186] Error getting master registration for 127.3.147.61:34697: OK, instance_id { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" instance_seqno: 1786527892070047 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 09:44:52.363307 29353 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 09:44:52.363438 29639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:52.363655 29639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [term 1 LEADER]: Becoming Leader. State: Replica: 25129173de6746d79239306724474817, State: Running, Role: LEADER
I20260812 09:44:52.364434 29639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } }
I20260812 09:44:52.365900 29660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
W20260812 09:44:52.366040 29672 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:52.366106 29672 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:52.366072 29660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:52.368369 29639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25129173de6746d79239306724474817. Latest consensus state: current_term: 1 leader_uuid: "25129173de6746d79239306724474817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.368523 29639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:52.381842 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.364924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50464 (local address 127.3.147.60:35123)
0812 09:44:52.365052 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.365056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.368494 (+ 3438us) server_negotiation.cc:407] Connection header received
0812 09:44:52.368741 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.368745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.368802 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.368845 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.369932 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.370689 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.371860 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.372227 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.372308 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.375541 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.375546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.375548 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.375574 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.377661 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.380350 (+ 2689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.380356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.380360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.380458 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.380787 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.380790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.380792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.381032 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.381149 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.381533 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.381658 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 09:44:52.385885 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.365336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.3.147.61:34697)
0812 09:44:52.365432 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.365436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.370125 (+ 4689us) server_negotiation.cc:407] Connection header received
0812 09:44:52.370164 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.370167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.370219 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.370291 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.373609 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.374406 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.375534 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.375887 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.375985 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.377787 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.377792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.377794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.377820 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.379533 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.383569 (+ 4036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.383576 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.383579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.383750 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.384305 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.384310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.384312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.384642 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.384840 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.385439 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.385627 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 09:44:52.392635 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:52.364621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41678 (local address 127.3.147.62:45719)
0812 09:44:52.364774 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.364778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.373546 (+ 8768us) server_negotiation.cc:407] Connection header received
0812 09:44:52.374581 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.374583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.374629 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.374694 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.378396 (+ 3702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.379222 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.386613 (+ 7391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:52.386978 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:52.387086 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:52.388867 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:52.388871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:52.388874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:52.388903 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:52.390650 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.391195 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.391198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.391200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.391288 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:52.391595 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:52.391599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:52.391601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:52.391829 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:52.391957 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:52.392357 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 09:44:52.392482 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:44:52.399641 29680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:52.400604 29680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:52.408067 29580 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [term 1 FOLLOWER]: Refusing update from remote peer 25129173de6746d79239306724474817: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:52.408774 29639 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:52.414942 29621 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25129173de6746d79239306724474817. Latest consensus state: current_term: 1 leader_uuid: "25129173de6746d79239306724474817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.415081 29621 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:52.416013 29494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [term 1 FOLLOWER]: Refusing update from remote peer 25129173de6746d79239306724474817: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:52.416574 29639 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:52.420012 29660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25129173de6746d79239306724474817. Latest consensus state: current_term: 1 leader_uuid: "25129173de6746d79239306724474817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.420161 29660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:52.426807 29639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25129173de6746d79239306724474817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.426949 29639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:52.428112 29660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25129173de6746d79239306724474817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.428414 29660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:52.428798 29681 mvcc.cc:205] Tried to move back new op lower bound from 7317618247292092416 to 7317618247130787840. Current Snapshot: MvccSnapshot[applied={T|T < 7317618247292092416}]
I20260812 09:44:52.429177 29685 mvcc.cc:205] Tried to move back new op lower bound from 7317618247292092416 to 7317618247130787840. Current Snapshot: MvccSnapshot[applied={T|T < 7317618247292092416}]
I20260812 09:44:52.434911 29673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25129173de6746d79239306724474817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.435902 29621 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25129173de6746d79239306724474817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25129173de6746d79239306724474817" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 45719 } } peers { permanent_uuid: "9b81a4f07dbc4feebe61533d6faa3255" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 34697 } } peers { permanent_uuid: "dc058334f1964977b4513980e01bb295" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 35123 } } }
I20260812 09:44:52.436041 29621 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:52.436511 29673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:52.437348 29682 mvcc.cc:205] Tried to move back new op lower bound from 7317618247292092416 to 7317618247130787840. Current Snapshot: MvccSnapshot[applied={T|T < 7317618247292092416}]
I20260812 09:44:52.441229 29680 catalog_manager.cc:1347] Generated new cluster ID: f4a892f0e32b43379f320f159160a286
I20260812 09:44:52.441298 29680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:52.475572 29680 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:52.476967 29680 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:52.493451 29680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25129173de6746d79239306724474817: Generated new TSK 0
I20260812 09:44:52.494218 29680 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 09:44:52.525517 29439 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.522221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.3.147.62:45719)
0812 09:44:52.522356 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.522362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.524329 (+ 1967us) server_negotiation.cc:407] Connection header received
0812 09:44:52.524542 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.524546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.524601 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.524657 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.525378 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41694: BlockingRecv error: recv got EOF from 127.0.0.1:41694 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 09:44:52.526975 29619 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.522966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.3.147.60:35123)
0812 09:44:52.523083 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.523088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.526026 (+ 2938us) server_negotiation.cc:407] Connection header received
0812 09:44:52.526167 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.526183 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.526237 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.526322 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.526835 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50470: BlockingRecv error: recv got EOF from 127.0.0.1:50470 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 09:44:52.527315 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.523607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51090 (local address 127.3.147.61:34697)
0812 09:44:52.523751 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.523755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.526338 (+ 2583us) server_negotiation.cc:407] Connection header received
0812 09:44:52.526521 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.526526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.526580 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.526634 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.527199 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51090: BlockingRecv error: recv got EOF from 127.0.0.1:51090 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 09:44:52.532850 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.531278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51098 (local address 127.3.147.61:34697)
0812 09:44:52.531385 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.531390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.531825 (+ 435us) server_negotiation.cc:407] Connection header received
0812 09:44:52.532034 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.532039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.532123 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.532204 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.532743 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51098: BlockingRecv error: recv got EOF from 127.0.0.1:51098 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 09:44:52.538108 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.536638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51114 (local address 127.3.147.61:34697)
0812 09:44:52.536749 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.536755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.537217 (+ 462us) server_negotiation.cc:407] Connection header received
0812 09:44:52.537397 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.537400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.537449 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.537498 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.538014 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51114: BlockingRecv error: recv got EOF from 127.0.0.1:51114 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 09:44:52.543820 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.542301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.3.147.61:34697)
0812 09:44:52.542402 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.542406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.542915 (+ 509us) server_negotiation.cc:407] Connection header received
0812 09:44:52.543117 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.543120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.543171 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.543223 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.543729 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51122: BlockingRecv error: recv got EOF from 127.0.0.1:51122 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 09:44:52.552336 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.550680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.3.147.61:34697)
0812 09:44:52.550843 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.550847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.551332 (+ 485us) server_negotiation.cc:407] Connection header received
0812 09:44:52.551519 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.551523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.551572 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.551623 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.552233 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51126: BlockingRecv error: recv got EOF from 127.0.0.1:51126 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260812 09:44:52.564677 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.562875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51128 (local address 127.3.147.61:34697)
0812 09:44:52.562997 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.563001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.563572 (+ 571us) server_negotiation.cc:407] Connection header received
0812 09:44:52.563813 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.563817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.563877 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.563936 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.564557 (+ 621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51128: BlockingRecv error: recv got EOF from 127.0.0.1:51128 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 09:44:52.584990 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.583564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.3.147.61:34697)
0812 09:44:52.583712 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.583717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.584003 (+ 286us) server_negotiation.cc:407] Connection header received
0812 09:44:52.584119 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.584122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.584229 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.584305 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.584832 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51136: BlockingRecv error: recv got EOF from 127.0.0.1:51136 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 09:44:52.620445 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.619078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.3.147.61:34697)
0812 09:44:52.619263 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.619268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.619416 (+ 148us) server_negotiation.cc:407] Connection header received
0812 09:44:52.619598 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.619602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.619658 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.619752 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.620314 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51142: BlockingRecv error: recv got EOF from 127.0.0.1:51142 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 09:44:52.687882 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.686583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.3.147.61:34697)
0812 09:44:52.686754 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.686759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.686918 (+ 159us) server_negotiation.cc:407] Connection header received
0812 09:44:52.687101 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.687105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.687162 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.687265 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.687750 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51156: BlockingRecv error: recv got EOF from 127.0.0.1:51156 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 09:44:52.819375 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:52.818106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.3.147.61:34697)
0812 09:44:52.818295 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:52.818300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:52.818430 (+ 130us) server_negotiation.cc:407] Connection header received
0812 09:44:52.818608 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:52.818611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:52.818667 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:52.818761 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:52.819253 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51168: BlockingRecv error: recv got EOF from 127.0.0.1:51168 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 09:44:53.078717 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:53.077633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.3.147.61:34697)
0812 09:44:53.077781 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:53.077786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:53.077897 (+ 111us) server_negotiation.cc:407] Connection header received
0812 09:44:53.078027 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:53.078030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:53.078091 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:53.078187 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:53.078605 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51176: BlockingRecv error: recv got EOF from 127.0.0.1:51176 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 09:44:53.348320 29635 catalog_manager.cc:1259] Loaded cluster ID: f4a892f0e32b43379f320f159160a286
I20260812 09:44:53.348462 29635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295: loading cluster ID for follower catalog manager: success
I20260812 09:44:53.350596 29635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295: acquiring CA information for follower catalog manager: success
I20260812 09:44:53.351848 29635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dc058334f1964977b4513980e01bb295: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 09:44:53.367484 29672 catalog_manager.cc:1259] Loaded cluster ID: f4a892f0e32b43379f320f159160a286
I20260812 09:44:53.367579 29672 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255: loading cluster ID for follower catalog manager: success
I20260812 09:44:53.369259 29672 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255: acquiring CA information for follower catalog manager: success
I20260812 09:44:53.370457 29672 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b81a4f07dbc4feebe61533d6faa3255: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 09:44:53.596625 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:53.592881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51192 (local address 127.3.147.61:34697)
0812 09:44:53.593159 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:53.593163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:53.593211 (+ 48us) server_negotiation.cc:407] Connection header received
0812 09:44:53.593340 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:53.593343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:53.593475 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:53.593570 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:53.593818 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.594368 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.595175 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.595404 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.595474 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:53.595754 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:53.596202 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:53.596294 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:53.596368 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 09:44:53.596417 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
I20260812 09:44:53.602840 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:53.599636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50472 (local address 127.3.147.60:35123)
0812 09:44:53.599856 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:53.599860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:53.599873 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:44:53.599946 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:53.599948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:53.600051 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:53.600237 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:53.600372 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.600854 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.601602 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.601858 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.601940 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:53.602192 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:53.602528 (+ 336us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:53.602614 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:53.602671 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 09:44:53.602719 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":75,"threads_started":1}
I20260812 09:44:53.610343 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:53.605832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.3.147.61:34697)
0812 09:44:53.606007 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:53.606012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:53.606087 (+ 75us) server_negotiation.cc:407] Connection header received
0812 09:44:53.606339 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:53.606342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:53.606396 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:53.606470 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:53.606719 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.607515 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.608940 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.609246 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.609349 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:53.609619 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:53.609689 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:53.609760 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:53.610161 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 09:44:53.610212 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 09:44:53.610354 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:53.605591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.3.147.62:45719)
0812 09:44:53.606110 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:53.606116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:53.606214 (+ 98us) server_negotiation.cc:407] Connection header received
0812 09:44:53.606449 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:53.606455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:53.606605 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:53.606715 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:53.607043 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.607565 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.608605 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.608850 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.608965 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:53.609397 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:53.609697 (+ 300us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:53.609800 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:53.609957 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 09:44:53.610038 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":67,"threads_started":1}
I20260812 09:44:53.610925 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:53.605673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50486 (local address 127.3.147.60:35123)
0812 09:44:53.605965 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:53.605970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:53.606207 (+ 237us) server_negotiation.cc:407] Connection header received
0812 09:44:53.606362 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:53.606365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:53.606410 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:53.606494 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:53.606831 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.607557 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.609406 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:53.609667 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:53.609785 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:53.610218 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:53.610304 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:53.610380 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:53.610753 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 09:44:53.610804 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212}
I20260812 09:44:53.613605 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 29353
I20260812 09:44:53.624758 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 29445
I20260812 09:44:53.632571 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 29533
2026-08-12T09:44:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3207 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 09:44:53 dist-test-slave-cbbz krb5kdc[29779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 09:44:53 dist-test-slave-cbbz krb5kdc[29779](info): set up 2 sockets
Aug 12 09:44:53 dist-test-slave-cbbz krb5kdc[29779](info): commencing operation
krb5kdc: starting...
W20260812 09:44:54.753999 3660 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/test-user.keytab.
Aug 12 09:44:54 dist-test-slave-cbbz krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T09:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T09:44:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.147.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
Entry for principal kudu/127.3.147.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab.
I20260812 09:44:54.881268 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:39905
--webserver_interface=127.3.147.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:39905,127.3.147.61:33217,127.3.147.60:33723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 09:44:54.991668 29794 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 09:44:54.991941 29794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:54.991964 29794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:54.992059 29794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:54.996510 29794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:54.996573 29794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:54.996593 29794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:54.996610 29794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:54.996651 29794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:55.001951 29794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:39905,127.3.147.61:33217,127.3.147.60:33723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.62.keytab
--principal=kudu/127.3.147.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.62:39905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.147.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:55.002983 29794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:55.004088 29794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:55.012789 29802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:55.012876 29804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:55.012890 29801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:55.012979 29794 server_base.cc:1034] running on GCE node
I20260812 09:44:55.013635 29794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:55.014739 29794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:55.015931 29794 hybrid_clock.cc:648] HybridClock initialized: now 1786527895015914 us; error 30 us; skew 500 ppm
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:55.018802 29794 init.cc:377] Logged in from keytab as kudu/127.3.147.62@KRBTEST.COM (short username kudu)
I20260812 09:44:55.019726 29794 webserver.cc:460] Webserver started at http://127.3.147.62:46173/ using document root <none> and password file <none>
I20260812 09:44:55.020332 29794 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:55.020423 29794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:55.020658 29794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:55.022655 29794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/data/instance:
uuid: "bc98ca64095642f497667f93dde97ed2"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "30a81b307f28bc90c8719cadc73697af"
server_key_iv: "d2dbb29d1206966460caa5b1d025567d"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.023222 29794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/wal/instance:
uuid: "bc98ca64095642f497667f93dde97ed2"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "30a81b307f28bc90c8719cadc73697af"
server_key_iv: "d2dbb29d1206966460caa5b1d025567d"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.026913 29794 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260812 09:44:55.029305 29813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:55.030529 29794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 09:44:55.030665 29794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/wal
uuid: "bc98ca64095642f497667f93dde97ed2"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "30a81b307f28bc90c8719cadc73697af"
server_key_iv: "d2dbb29d1206966460caa5b1d025567d"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.030781 29794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:55.058691 29794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:55.061738 29794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:55.061956 29794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:55.068930 29794 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.62:39905
I20260812 09:44:55.069242 29869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.62:39905 every 8 connection(s)
I20260812 09:44:55.069891 29794 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/data/info.pb
I20260812 09:44:55.072705 29870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:55.076469 29870 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:55.077673 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 29794
I20260812 09:44:55.077776 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-0/wal/instance
I20260812 09:44:55.078056 3660 external_mini_cluster.cc:1428] Setting key 1a82311a550296bae25bb687ed1cbd85
WARNING: no policy specified for kudu/127.3.147.61@KRBTEST.COM; defaulting to no policy
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.61@KRBTEST.COM" created.
I20260812 09:44:55.092891 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.078839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:39905 (local address 127.0.0.1:55220)
0812 09:44:55.079442 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.079490 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.080713 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.081422 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.081430 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.081948 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.082335 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.082357 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.083572 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.083579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.084958 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.084973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.085625 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.085631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.085773 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.086391 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.086415 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.088048 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.089929 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.089938 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.089944 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.090271 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.090590 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.090593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.090595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.090699 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.091236 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.091242 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.091374 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.091529 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.091903 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":118,"spinlock_wait_cycles":52608,"thread_start_us":143,"threads_started":1}
I20260812 09:44:55.092891 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.078839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.3.147.62:39905)
0812 09:44:55.079327 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.079347 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.079605 (+ 258us) server_negotiation.cc:407] Connection header received
0812 09:44:55.081029 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.081034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.081212 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.081361 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.082498 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.083435 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.085080 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.085443 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.085563 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.088246 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.088251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.088260 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.088281 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.089829 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.090381 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.090387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.090388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.090491 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.090795 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.090800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.090802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.091122 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.091267 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.091598 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.091856 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":314,"thread_start_us":143,"threads_started":1}
I20260812 09:44:55.095430 29870 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:55.096206 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.095864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34066)
0812 09:44:55.096016 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.096111 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 09:44:55.096418 29817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
I20260812 09:44:55.097298 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.097031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34080)
0812 09:44:55.097170 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.097226 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 09:44:55.097470 29870 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:33217: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab.
Entry for principal kudu/127.3.147.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab.
I20260812 09:44:55.105675 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.61:33217
--webserver_interface=127.3.147.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab
--principal=kudu/127.3.147.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:39905,127.3.147.61:33217,127.3.147.60:33723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:44:55.146741 29870 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } attempt: 1
I20260812 09:44:55.147771 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.147360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34088)
0812 09:44:55.147558 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.147657 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 09:44:55.148571 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.148340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34100)
0812 09:44:55.148453 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.148506 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 09:44:55.148723 29870 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:33217: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
W20260812 09:44:55.216398 29877 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 09:44:55.216655 29877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:55.216679 29877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:55.216733 29877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 09:44:55.216887 29870 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } attempt: 2
I20260812 09:44:55.217815 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.217486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34104)
0812 09:44:55.217655 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.217725 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 09:44:55.218531 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.218314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34120)
0812 09:44:55.218420 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.218470 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 09:44:55.218658 29870 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.61:33217: Network error: Client connection negotiation failed: client connection to 127.3.147.61:33217: connect: Connection refused (error 111)
W20260812 09:44:55.221585 29877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:55.221639 29877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:55.221658 29877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:55.221675 29877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:55.221696 29877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:55.226861 29877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:39905,127.3.147.61:33217,127.3.147.60:33723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.61.keytab
--principal=kudu/127.3.147.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.61:33217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.147.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:55.227806 29877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:55.228840 29877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:55.237313 29886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:55.237339 29883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:55.237339 29882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:55.237476 29877 server_base.cc:1034] running on GCE node
I20260812 09:44:55.238157 29877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:55.239190 29877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:55.240389 29877 hybrid_clock.cc:648] HybridClock initialized: now 1786527895240374 us; error 27 us; skew 500 ppm
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:55.243136 29877 init.cc:377] Logged in from keytab as kudu/127.3.147.61@KRBTEST.COM (short username kudu)
I20260812 09:44:55.243970 29877 webserver.cc:460] Webserver started at http://127.3.147.61:41455/ using document root <none> and password file <none>
I20260812 09:44:55.244556 29877 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:55.244653 29877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:55.244877 29877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:55.246933 29877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/data/instance:
uuid: "e1dd3fbd99f44111aaa7cd7f3223414b"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "cee6b6790fb4a5a9399698494d61d44f"
server_key_iv: "806f0771ed9b7b5660040e6fa84de396"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.247509 29877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/wal/instance:
uuid: "e1dd3fbd99f44111aaa7cd7f3223414b"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "cee6b6790fb4a5a9399698494d61d44f"
server_key_iv: "806f0771ed9b7b5660040e6fa84de396"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.251209 29877 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 09:44:55.253593 29895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:55.254949 29877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 09:44:55.255088 29877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/wal
uuid: "e1dd3fbd99f44111aaa7cd7f3223414b"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "cee6b6790fb4a5a9399698494d61d44f"
server_key_iv: "806f0771ed9b7b5660040e6fa84de396"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.255198 29877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:55.274091 29877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:55.277138 29877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:55.277369 29877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:55.284252 29877 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.61:33217
I20260812 09:44:55.284562 29956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.61:33217 every 8 connection(s)
I20260812 09:44:55.285282 29877 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/data/info.pb
I20260812 09:44:55.288125 29957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:55.291378 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 29877
I20260812 09:44:55.291543 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-1/wal/instance
I20260812 09:44:55.291844 3660 external_mini_cluster.cc:1428] Setting key e4cc9c53259e8f8313bcb263674bfe65
I20260812 09:44:55.292182 29957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.147.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.147.60@KRBTEST.COM" created.
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:55.309809 29870 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } attempt: 3
I20260812 09:44:55.309973 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.294247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.3.147.62:39905)
0812 09:44:55.294434 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.294441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.295374 (+ 933us) server_negotiation.cc:407] Connection header received
0812 09:44:55.296583 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.296591 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.296698 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.296811 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.298165 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.299199 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.300400 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.300861 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.301023 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.305968 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.305971 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.305973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.306002 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.307582 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.308199 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.308206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.308210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.308306 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.308686 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.308692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.308696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.308968 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.309080 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.309645 (+ 565us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.309810 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 09:44:55.310885 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.294653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:39905 (local address 127.0.0.1:55230)
0812 09:44:55.295191 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.295271 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.296271 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.297107 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.297118 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.297516 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.297990 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.298003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.299345 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.299348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.300270 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.300279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.301057 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.301059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.301196 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.301830 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.301847 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.305766 (+ 3919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.307728 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.307735 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.307752 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.308026 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.308420 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.308423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.308425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.308571 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.309076 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.309082 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.309339 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.309495 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.309795 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":141,"threads_started":1}
I20260812 09:44:55.312654 29957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.147.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab.
Entry for principal kudu/127.3.147.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab.
I20260812 09:44:55.323366 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.313199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34140)
0812 09:44:55.313447 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.313461 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.313540 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.313766 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.313768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.313946 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.314024 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.314029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.314801 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.314804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.315316 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.315324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.315842 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.315845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.315970 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.316564 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.316584 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.319076 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.321489 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.321495 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.321497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.321739 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.322329 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.322332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.322333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.322405 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.322836 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.322839 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.322913 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.323018 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.323151 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":21}
I20260812 09:44:55.323369 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.310248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34134)
0812 09:44:55.310367 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.310383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.310468 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.311658 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.311660 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.311838 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.311936 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.311941 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.313464 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.313469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.314986 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.315000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.315900 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.315904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.316020 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.316606 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.316620 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.318575 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.321599 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.321603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.321605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.321876 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.322135 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.322138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.322140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.322228 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.322792 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.322798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.322923 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.323043 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.323196 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 09:44:55.323416 29963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.313222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.3.147.61:33217)
0812 09:44:55.313517 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.313525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.313538 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:44:55.313617 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.313622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.313656 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.313757 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.314135 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.314676 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.315430 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.315722 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.315845 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.319314 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.319324 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.319326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.319348 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.321377 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.321876 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.321881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.321882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.322046 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.322521 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.322524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.322525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.322726 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.322806 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.323116 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.323244 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":230,"thread_start_us":109,"threads_started":1}
I20260812 09:44:55.323813 3660 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
/tmp/dist-test-tasknDSHUD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.60:33723
--webserver_interface=127.3.147.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab
--principal=kudu/127.3.147.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.147.20:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.147.62:39905,127.3.147.61:33217,127.3.147.60:33723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 09:44:55.324116 29962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.310445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.3.147.61:33217)
0812 09:44:55.310851 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.310865 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.310890 (+ 25us) server_negotiation.cc:407] Connection header received
0812 09:44:55.310936 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.310944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.311509 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.311691 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.312062 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.313303 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.315155 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.315762 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.315900 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.318727 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.318737 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.318767 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.318795 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.321462 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.321971 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.321976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.321981 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.322051 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.322312 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.322315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.322316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.322623 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.322760 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.323726 (+ 966us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.323865 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":142,"threads_started":1}
I20260812 09:44:55.325619 29870 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:55.325821 29957 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:55.326390 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.326088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33350)
0812 09:44:55.326259 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.326305 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 09:44:55.326591 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.326259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33358)
0812 09:44:55.326436 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.326515 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260812 09:44:55.326774 29897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
I20260812 09:44:55.327239 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.326945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33374)
0812 09:44:55.327101 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.327162 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 09:44:55.327467 29870 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:33723: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
I20260812 09:44:55.327608 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.327405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33378)
0812 09:44:55.327495 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.327547 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 09:44:55.327754 29957 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:33723: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
I20260812 09:44:55.358634 29870 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } attempt: 1
I20260812 09:44:55.359515 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.359199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33392)
0812 09:44:55.359372 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.359435 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 09:44:55.360461 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.360200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33406)
0812 09:44:55.360340 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.360398 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 09:44:55.360615 29870 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:33723: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
I20260812 09:44:55.377017 29957 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } attempt: 1
I20260812 09:44:55.377807 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.377511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33408)
0812 09:44:55.377674 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.377732 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 09:44:55.378607 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.378391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33416)
0812 09:44:55.378502 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.378552 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 09:44:55.378793 29957 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:33723: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
I20260812 09:44:55.435796 29870 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } attempt: 2
I20260812 09:44:55.436743 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.436372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33432)
0812 09:44:55.436567 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.436644 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 09:44:55.437510 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.437263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33436)
0812 09:44:55.437387 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.437441 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 09:44:55.437690 29870 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:33723: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
W20260812 09:44:55.442575 29966 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 09:44:55.442822 29966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 09:44:55.442847 29966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 09:44:55.442889 29966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 09:44:55.446887 29966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 09:44:55.446938 29966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 09:44:55.446955 29966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 09:44:55.446996 29966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 09:44:55.447031 29966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 09:44:55.446971 29957 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } attempt: 2
I20260812 09:44:55.447746 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.447480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33438)
0812 09:44:55.447613 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.447668 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 09:44:55.448589 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.448374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33442)
0812 09:44:55.448483 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.448535 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 09:44:55.448747 29957 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.147.60:33723: Network error: Client connection negotiation failed: client connection to 127.3.147.60:33723: connect: Connection refused (error 111)
I20260812 09:44:55.452804 29966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.147.20:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.147.62:39905,127.3.147.61:33217,127.3.147.60:33723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/krb5kdc/kudu_127.3.147.60.keytab
--principal=kudu/127.3.147.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.147.60:33723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.147.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 08:09:26 UTC on ccycloud.quasar-kitgea.root.comops.site
build id 4582138
I20260812 09:44:55.453832 29966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 09:44:55.454851 29966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 09:44:55.463584 29977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 09:44:55.463577 29973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:55.463711 29966 server_base.cc:1034] running on GCE node
W20260812 09:44:55.463579 29974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 09:44:55.464414 29966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 09:44:55.465567 29966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 09:44:55.466742 29966 hybrid_clock.cc:648] HybridClock initialized: now 1786527895466730 us; error 29 us; skew 500 ppm
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 09:44:55.469611 29966 init.cc:377] Logged in from keytab as kudu/127.3.147.60@KRBTEST.COM (short username kudu)
I20260812 09:44:55.470402 29966 webserver.cc:460] Webserver started at http://127.3.147.60:40947/ using document root <none> and password file <none>
I20260812 09:44:55.470898 29966 fs_manager.cc:359] Metadata directory not provided
I20260812 09:44:55.470954 29966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 09:44:55.471117 29966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 09:44:55.473126 29966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/data/instance:
uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "62321caef104013f993dc54947fed1c7"
server_key_iv: "4f4e17bb33e965a8377744a10bde6a79"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.473640 29966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/wal/instance:
uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "62321caef104013f993dc54947fed1c7"
server_key_iv: "4f4e17bb33e965a8377744a10bde6a79"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.477387 29966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 09:44:55.479648 29985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 09:44:55.480912 29966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 09:44:55.481055 29966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/data,/tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/wal
uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed"
format_stamp: "Formatted at 2026-08-12 09:44:55 on dist-test-slave-cbbz"
server_key: "62321caef104013f993dc54947fed1c7"
server_key_iv: "4f4e17bb33e965a8377744a10bde6a79"
server_key_version: "encryptionkey@0"
I20260812 09:44:55.481166 29966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 09:44:55.506547 29966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 09:44:55.509541 29966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 09:44:55.509749 29966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 09:44:55.517071 29966 rpc_server.cc:307] RPC server started. Bound to: 127.3.147.60:33723
I20260812 09:44:55.517484 30045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.147.60:33723 every 8 connection(s)
I20260812 09:44:55.518108 29966 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/data/info.pb
I20260812 09:44:55.521150 30046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 09:44:55.525318 30046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:55.525890 3660 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu as pid 29966
I20260812 09:44:55.526062 3660 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknDSHUD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786527603645036-3660-0/minicluster-data/master-2/wal/instance
I20260812 09:44:55.526372 3660 external_mini_cluster.cc:1428] Setting key 48183684db2e2b15b317ef636dd4fbed
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:44:55.536876 29870 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } attempt: 3
I20260812 09:44:55.540190 29957 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } attempt: 3
I20260812 09:44:55.540270 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.527532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.3.147.62:39905)
0812 09:44:55.527671 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.527676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.528570 (+ 894us) server_negotiation.cc:407] Connection header received
0812 09:44:55.529416 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.529419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.529482 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.529568 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.530965 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.531525 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.532591 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.532810 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.532922 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.536072 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.536076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.536079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.536120 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.537890 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.538537 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.538540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.538542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.538628 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.539016 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.539019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.539021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.539278 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.539389 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.539918 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.540065 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 09:44:55.541288 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.527831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.62:39905 (local address 127.0.0.1:55242)
0812 09:44:55.528407 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.528468 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.529184 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.530005 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.530009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.530352 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.530785 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.530801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.531646 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.531652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.532466 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.532475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.532973 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.532977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.533172 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.533834 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.533866 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.535884 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.538017 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.538030 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.538043 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.538381 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.538740 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.538744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.538746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.538904 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.539405 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.539413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.539685 (+ 272us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.539874 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.540244 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260812 09:44:55.542248 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.527831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.3.147.60:33723)
0812 09:44:55.528354 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.528372 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.528409 (+ 37us) server_negotiation.cc:407] Connection header received
0812 09:44:55.529160 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.529179 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.529421 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.529614 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.530545 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.531636 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.532671 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.533120 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.533269 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.536569 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.536574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.536583 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.536605 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.539205 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.539772 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.539777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.539779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.539860 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.540221 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.540224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.540225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.540639 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.541784 (+ 1145us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.541920 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.542062 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":406,"thread_start_us":180,"threads_started":1}
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.62@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:44:55.543044 30046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.61@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:44:55.547367 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.537424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33464)
0812 09:44:55.537554 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.537572 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.537653 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.538132 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.538135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.538283 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.538358 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.538363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.539285 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.539289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.540161 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.540452 (+ 291us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.541034 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.541037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.541143 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.541669 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.541687 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.543161 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.545539 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.545545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.545550 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.545800 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.546379 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.546382 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.546384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.546475 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.546908 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.546911 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.546994 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.547070 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.547188 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 09:44:55.548674 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.537521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.3.147.60:33723)
0812 09:44:55.537870 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.537879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.537894 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:44:55.537928 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.537934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.537991 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.538093 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.538474 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.539161 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.540571 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.540904 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.541517 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.543299 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.543303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.543305 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.543328 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.545415 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.546149 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.546153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.546154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.546272 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.546591 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.546593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.546595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.546791 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.548006 (+ 1215us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.548084 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.548474 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":120,"threads_started":1}
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
I20260812 09:44:55.552313 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.541200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33472 (local address 127.3.147.60:33723)
0812 09:44:55.541705 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.541716 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.541728 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:44:55.541760 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.541762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.541821 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.542432 (+ 611us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.542531 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.543622 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.544787 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.545433 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.545564 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.547885 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.547889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.547891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.547915 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.550254 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.550734 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.550738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.550739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.550819 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.551399 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.551405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.551406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.551612 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.551713 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.552037 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.552168 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":88,"threads_started":1}
I20260812 09:44:55.552333 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.540732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33472)
0812 09:44:55.540911 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.540925 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.541003 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.541978 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.541981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.542179 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.542302 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.542313 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.543752 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.543757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.544640 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.544655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.545572 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.545576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.545673 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.546135 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.546150 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.547630 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.550376 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.550380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.550382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.550635 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.551179 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.551183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.551185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.551280 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.551730 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.551735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.551851 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.551968 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.552126 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 09:44:55.552523 29870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2: Bootstrap starting.
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.62@KRBTEST.COM
I20260812 09:44:55.555473 29957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b: Bootstrap starting.
I20260812 09:44:55.555553 29870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:55.556934 29870 log.cc:824] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:55.557345 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.546035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55256 (local address 127.3.147.62:39905)
0812 09:44:55.546787 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.546792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.546807 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:44:55.546850 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.546853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.546890 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.546965 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.547808 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.548575 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.550164 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.550473 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.550576 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.553392 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.553395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.553398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.553422 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.555102 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.555596 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.555608 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.555610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.555695 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.556026 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.556029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.556030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.556322 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.556481 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.557078 (+ 597us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.557204 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656}
I20260812 09:44:55.558917 29957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:55.559839 29957 log.cc:824] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:55.560447 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.543792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.61:33217 (local address 127.0.0.1:34148)
0812 09:44:55.543932 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.543946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.544013 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.544483 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.544487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.544670 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.544784 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.544791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.545865 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.545871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.546711 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.546719 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.547478 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.547483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.547607 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.548020 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.548036 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.556237 (+ 8201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.558540 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.558543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.558545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.558805 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.559186 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.559189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.559191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.559287 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.559874 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.559877 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.559987 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.560128 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.560290 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 09:44:55.560478 29963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.544044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.3.147.61:33217)
0812 09:44:55.544198 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.544203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.544216 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:44:55.544250 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.544253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.544293 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.544356 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.544923 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.545496 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.546854 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.547226 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.547302 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.556403 (+ 9101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.556407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.556409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.556434 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.558414 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.558932 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.558937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.558939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.559024 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.559416 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.559421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.559423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.559763 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.559881 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.560209 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.560331 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 09:44:55.561301 30046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } has no permanent_uuid. Determining permanent_uuid...
I20260812 09:44:55.562521 29870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2: No bootstrap required, opened a new log
I20260812 09:44:55.563544 29957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b: No bootstrap required, opened a new log
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.147.61@KRBTEST.COM
I20260812 09:44:55.566833 29870 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } }
I20260812 09:44:55.567085 29870 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc98ca64095642f497667f93dde97ed2, State: Initialized, Role: FOLLOWER
I20260812 09:44:55.567415 29870 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } }
Aug 12 09:44:55 dist-test-slave-cbbz krb5kdc[29779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786527895, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.147.60@KRBTEST.COM for kudu/127.3.147.60@KRBTEST.COM
I20260812 09:44:55.567947 30058 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.568176 30058 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:55.568850 29870 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:55.568776 29957 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } }
I20260812 09:44:55.569005 29957 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1dd3fbd99f44111aaa7cd7f3223414b, State: Initialized, Role: FOLLOWER
I20260812 09:44:55.569347 29957 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } }
I20260812 09:44:55.569900 30059 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.570133 30059 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:55.570712 29957 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:55.571856 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.561715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.147.60:33723 (local address 127.0.0.1:33474)
0812 09:44:55.561850 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 09:44:55.561865 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 09:44:55.561940 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 09:44:55.562793 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 09:44:55.562797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 09:44:55.562967 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 09:44:55.563059 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.563066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.563990 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.563992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.564889 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 09:44:55.564900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.565796 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.565800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 09:44:55.566141 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.566549 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 09:44:55.566563 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 09:44:55.568198 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 09:44:55.570129 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.570132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.570133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.570360 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.570710 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 09:44:55.570716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 09:44:55.570721 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 09:44:55.570821 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 09:44:55.571343 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 09:44:55.571348 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 09:44:55.571461 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 09:44:55.571562 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 09:44:55.571708 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 09:44:55.572237 29962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.558582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.3.147.61:33217)
0812 09:44:55.558705 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.558710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.558725 (+ 15us) server_negotiation.cc:407] Connection header received
0812 09:44:55.558844 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.558847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.558888 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.558981 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.559687 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.560515 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.561490 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.561853 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.561954 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.565118 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.565122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.565124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.565148 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.566765 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.568700 (+ 1935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.568704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.568706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.569162 (+ 456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.570698 (+ 1536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.570701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.570703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.570950 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.571148 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.571915 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.572043 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:44:55.572593 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.562095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.3.147.60:33723)
0812 09:44:55.562217 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.562221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.562233 (+ 12us) server_negotiation.cc:407] Connection header received
0812 09:44:55.562265 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.562268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.562309 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.562381 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.563175 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.563881 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.565009 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.565326 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.565701 (+ 375us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.568338 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.568341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.568344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.568366 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.569995 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.570471 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.570476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.570479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.570576 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.570975 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.570978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.570980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.571203 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.571307 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.572356 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.572468 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:55.575889 30046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed: Bootstrap starting.
W20260812 09:44:55.576356 30084 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:55.576476 30084 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:55.576814 3660 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 09:44:55.577895 29843 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bc98ca64095642f497667f93dde97ed2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55256
I20260812 09:44:55.578111 29843 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 09:44:55.578222 29843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:55.579649 29843 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } }
I20260812 09:44:55.580590 29843 leader_election.cc:288] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [CANDIDATE]: Term 1 election: Requested vote from peers e1dd3fbd99f44111aaa7cd7f3223414b (127.3.147.61:33217), a85d9dd9d60d4ed088e9ab9d96c026ed (127.3.147.60:33723)
I20260812 09:44:55.581024 29927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc98ca64095642f497667f93dde97ed2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b"
I20260812 09:44:55.581259 29927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:55.582396 30046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed: Neither blocks nor log segments found. Creating new log.
I20260812 09:44:55.582643 29927 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc98ca64095642f497667f93dde97ed2 in term 1.
I20260812 09:44:55.583045 29817 leader_election.cc:302] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc98ca64095642f497667f93dde97ed2, e1dd3fbd99f44111aaa7cd7f3223414b; no voters:
I20260812 09:44:55.583366 30058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 09:44:55.583344 30046 log.cc:824] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed: Log is configured to *not* fsync() on all Append() calls
I20260812 09:44:55.583590 30058 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [term 1 LEADER]: Becoming Leader. State: Replica: bc98ca64095642f497667f93dde97ed2, State: Running, Role: LEADER
I20260812 09:44:55.583925 30058 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } }
I20260812 09:44:55.585664 30058 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc98ca64095642f497667f93dde97ed2. Latest consensus state: current_term: 1 leader_uuid: "bc98ca64095642f497667f93dde97ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.585793 30058 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:55.577553 29794 master_runner.cc:186] Error getting master registration for 127.3.147.60:33723: OK, instance_id { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" instance_seqno: 1786527895514912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 09:44:55.586354 29794 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 09:44:55.583765 29877 master_runner.cc:186] Error getting master registration for 127.3.147.60:33723: OK, instance_id { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" instance_seqno: 1786527895514912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 09:44:55.587489 29877 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 09:44:55.589373 30071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:55.589440 30071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:55.589762 30095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 09:44:55.590358 30046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed: No bootstrap required, opened a new log
I20260812 09:44:55.590636 30095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 09:44:55.595177 30046 raft_consensus.cc:348] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } }
I20260812 09:44:55.595409 30046 raft_consensus.cc:729] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85d9dd9d60d4ed088e9ab9d96c026ed, State: Initialized, Role: FOLLOWER
I20260812 09:44:55.595729 30046 consensus_queue.cc:260] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } }
I20260812 09:44:55.597204 30046 sys_catalog.cc:564] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [sys.catalog]: configured and running, proceeding with master startup.
I20260812 09:44:55.597345 30016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc98ca64095642f497667f93dde97ed2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed"
I20260812 09:44:55.597594 30016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 09:44:55.598963 30016 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc98ca64095642f497667f93dde97ed2 in term 1.
I20260812 09:44:55.599355 30015 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [term 1 FOLLOWER]: Refusing update from remote peer bc98ca64095642f497667f93dde97ed2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:55.600674 30058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:55.601313 29927 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [term 1 FOLLOWER]: Refusing update from remote peer bc98ca64095642f497667f93dde97ed2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 09:44:55.602080 30058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 09:44:55.603392 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.582918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55270 (local address 127.3.147.62:39905)
0812 09:44:55.583028 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.583033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.588810 (+ 5777us) server_negotiation.cc:407] Connection header received
0812 09:44:55.589098 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.589102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.589162 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.589215 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.590815 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.591666 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.593566 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.593911 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.593994 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.596876 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.596880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.596883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.596920 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.601347 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.601889 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.601893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.601895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.601990 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.602318 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.602324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.602326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.602571 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.602715 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.603071 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.603208 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 09:44:55.604277 30111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 09:44:55.604349 30111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 09:44:55.611608 30059 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc98ca64095642f497667f93dde97ed2. Latest consensus state: current_term: 1 leader_uuid: "bc98ca64095642f497667f93dde97ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.611760 30059 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:55.612885 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.600516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33480 (local address 127.3.147.60:33723)
0812 09:44:55.600645 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.600652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.604060 (+ 3408us) server_negotiation.cc:407] Connection header received
0812 09:44:55.604282 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.604285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.604332 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.604383 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.605322 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.606005 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.607044 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.607358 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.607432 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.609041 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.609046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.609052 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.609079 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.610830 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.611334 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.611340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.611345 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.611438 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.611860 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.611864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.611869 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.612084 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.612253 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.612615 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.612734 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 09:44:55.613646 29962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:55.583292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34152 (local address 127.3.147.61:33217)
0812 09:44:55.583396 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.583400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.602501 (+ 19101us) server_negotiation.cc:407] Connection header received
0812 09:44:55.602703 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.602707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.602758 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.602807 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.603654 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.604403 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.605552 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:55.605886 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:55.605956 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:55.607739 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 09:44:55.607743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 09:44:55.607746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 09:44:55.607775 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 09:44:55.610037 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.610587 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.610591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.610592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.610680 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 09:44:55.612677 (+ 1997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 09:44:55.612681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 09:44:55.612683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 09:44:55.612897 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 09:44:55.612998 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:55.613368 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 09:44:55.613502 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 09:44:55.617213 30058 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc98ca64095642f497667f93dde97ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.617344 30058 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:55.618765 30115 mvcc.cc:205] Tried to move back new op lower bound from 7317618260368326656 to 7317618260315217920. Current Snapshot: MvccSnapshot[applied={T|T < 7317618260368326656}]
I20260812 09:44:55.619330 30098 mvcc.cc:205] Tried to move back new op lower bound from 7317618260368326656 to 7317618260315217920. Current Snapshot: MvccSnapshot[applied={T|T < 7317618260368326656}]
I20260812 09:44:55.620400 30114 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc98ca64095642f497667f93dde97ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.620551 30114 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:55.620652 30114 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc98ca64095642f497667f93dde97ed2. Latest consensus state: current_term: 1 leader_uuid: "bc98ca64095642f497667f93dde97ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.620746 30114 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:55.620836 30114 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc98ca64095642f497667f93dde97ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.620918 30114 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:55.623461 30095 catalog_manager.cc:1347] Generated new cluster ID: 487149e6e3ed4a6d86530b5368148a35
I20260812 09:44:55.623546 30095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 09:44:55.624866 30092 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc98ca64095642f497667f93dde97ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.625002 30092 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2 [sys.catalog]: This master's current role is: LEADER
I20260812 09:44:55.626070 30059 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc98ca64095642f497667f93dde97ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc98ca64095642f497667f93dde97ed2" member_type: VOTER last_known_addr { host: "127.3.147.62" port: 39905 } } peers { permanent_uuid: "e1dd3fbd99f44111aaa7cd7f3223414b" member_type: VOTER last_known_addr { host: "127.3.147.61" port: 33217 } } peers { permanent_uuid: "a85d9dd9d60d4ed088e9ab9d96c026ed" member_type: VOTER last_known_addr { host: "127.3.147.60" port: 33723 } } }
I20260812 09:44:55.626195 30059 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 09:44:55.626766 30117 mvcc.cc:205] Tried to move back new op lower bound from 7317618260368326656 to 7317618260315217920. Current Snapshot: MvccSnapshot[applied={T|T < 7317618260368326656}]
I20260812 09:44:55.643831 30095 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 09:44:55.645318 30095 catalog_manager.cc:1504] Loading token signing keys...
I20260812 09:44:55.658486 30095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc98ca64095642f497667f93dde97ed2: Generated new TSK 0
I20260812 09:44:55.659605 30095 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 09:44:55.691658 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.688997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.3.147.61:33217)
0812 09:44:55.689131 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.689136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.690490 (+ 1354us) server_negotiation.cc:407] Connection header received
0812 09:44:55.690712 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.690716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.690767 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.690822 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.691501 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34156: BlockingRecv error: recv got EOF from 127.0.0.1:34156 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 09:44:55.693131 30055 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.689855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.3.147.60:33723)
0812 09:44:55.689972 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.689977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.692210 (+ 2233us) server_negotiation.cc:407] Connection header received
0812 09:44:55.692390 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.692393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.692445 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.692496 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.693001 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33496: BlockingRecv error: recv got EOF from 127.0.0.1:33496 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 09:44:55.695082 29872 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.693638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55276 (local address 127.3.147.62:39905)
0812 09:44:55.693748 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.693753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.694184 (+ 431us) server_negotiation.cc:407] Connection header received
0812 09:44:55.694363 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.694366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.694416 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.694477 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.694966 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55276: BlockingRecv error: recv got EOF from 127.0.0.1:55276 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 09:44:55.699539 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.697908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.3.147.61:33217)
0812 09:44:55.698004 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.698007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.698493 (+ 486us) server_negotiation.cc:407] Connection header received
0812 09:44:55.698837 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.698840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.698921 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.698981 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.699444 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34172: BlockingRecv error: recv got EOF from 127.0.0.1:34172 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 09:44:55.704476 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.702929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.3.147.61:33217)
0812 09:44:55.703027 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.703030 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.703536 (+ 506us) server_negotiation.cc:407] Connection header received
0812 09:44:55.703729 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.703732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.703785 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.703833 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.704380 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34174: BlockingRecv error: recv got EOF from 127.0.0.1:34174 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 09:44:55.709898 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.708387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.3.147.61:33217)
0812 09:44:55.708482 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.708485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.709006 (+ 521us) server_negotiation.cc:407] Connection header received
0812 09:44:55.709193 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.709198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.709249 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.709297 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.709774 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34190: BlockingRecv error: recv got EOF from 127.0.0.1:34190 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 09:44:55.717262 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.715778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.3.147.61:33217)
0812 09:44:55.715900 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.715904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.716431 (+ 527us) server_negotiation.cc:407] Connection header received
0812 09:44:55.716600 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.716603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.716660 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.716708 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.717167 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34206: BlockingRecv error: recv got EOF from 127.0.0.1:34206 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 09:44:55.728844 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.727229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.3.147.61:33217)
0812 09:44:55.727331 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.727335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.727900 (+ 565us) server_negotiation.cc:407] Connection header received
0812 09:44:55.728085 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.728088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.728171 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.728228 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.728753 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34208: BlockingRecv error: recv got EOF from 127.0.0.1:34208 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 09:44:55.749120 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.747220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.3.147.61:33217)
0812 09:44:55.747348 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.747352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.747998 (+ 646us) server_negotiation.cc:407] Connection header received
0812 09:44:55.748264 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.748269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.748325 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.748385 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.748996 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34218: BlockingRecv error: recv got EOF from 127.0.0.1:34218 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 09:44:55.791046 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.789283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.3.147.61:33217)
0812 09:44:55.789417 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.789422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.790044 (+ 622us) server_negotiation.cc:407] Connection header received
0812 09:44:55.790254 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.790258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.790312 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.790366 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.790925 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34222: BlockingRecv error: recv got EOF from 127.0.0.1:34222 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 09:44:55.859002 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.857741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.3.147.61:33217)
0812 09:44:55.857904 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.857910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.858026 (+ 116us) server_negotiation.cc:407] Connection header received
0812 09:44:55.858209 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.858214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.858276 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.858381 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.858864 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34228: BlockingRecv error: recv got EOF from 127.0.0.1:34228 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 09:44:55.990303 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:55.989135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.3.147.61:33217)
0812 09:44:55.989334 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:55.989339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:55.989427 (+ 88us) server_negotiation.cc:407] Connection header received
0812 09:44:55.989552 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:55.989555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:55.989608 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:55.989697 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:55.990152 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34238: BlockingRecv error: recv got EOF from 127.0.0.1:34238 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260812 09:44:56.249673 29962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 09:44:56.248474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.3.147.61:33217)
0812 09:44:56.248623 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:56.248629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:56.248730 (+ 101us) server_negotiation.cc:407] Connection header received
0812 09:44:56.248906 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:56.248910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:56.248970 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:56.249064 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 09:44:56.249541 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34250: BlockingRecv error: recv got EOF from 127.0.0.1:34250 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 09:44:56.577894 30084 catalog_manager.cc:1259] Loaded cluster ID: 487149e6e3ed4a6d86530b5368148a35
I20260812 09:44:56.577987 30084 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b: loading cluster ID for follower catalog manager: success
I20260812 09:44:56.579638 30084 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b: acquiring CA information for follower catalog manager: success
I20260812 09:44:56.580757 30084 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e1dd3fbd99f44111aaa7cd7f3223414b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 09:44:56.605701 30111 catalog_manager.cc:1259] Loaded cluster ID: 487149e6e3ed4a6d86530b5368148a35
I20260812 09:44:56.605784 30111 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed: loading cluster ID for follower catalog manager: success
I20260812 09:44:56.607589 30111 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed: acquiring CA information for follower catalog manager: success
I20260812 09:44:56.608860 30111 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a85d9dd9d60d4ed088e9ab9d96c026ed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 09:44:56.767490 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:56.763775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.3.147.61:33217)
0812 09:44:56.764031 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:56.764036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:56.764074 (+ 38us) server_negotiation.cc:407] Connection header received
0812 09:44:56.764269 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:56.764272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:56.764400 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:56.764515 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:56.764758 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.765290 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.766105 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.766350 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.766437 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:56.766735 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:56.767173 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:56.767266 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:56.767317 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 09:44:56.767365 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":102,"threads_started":1}
I20260812 09:44:56.774410 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:56.770645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.3.147.60:33723)
0812 09:44:56.770898 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:56.770902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:56.770915 (+ 13us) server_negotiation.cc:407] Connection header received
0812 09:44:56.770994 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:56.770997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:56.771193 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:56.771331 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:56.771577 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.772061 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.773000 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.773216 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.773330 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:56.773610 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:56.773994 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:56.774082 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:56.774164 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 09:44:56.774218 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260812 09:44:56.781785 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:56.777528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33512 (local address 127.3.147.60:33723)
0812 09:44:56.777628 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:56.777632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:56.778064 (+ 432us) server_negotiation.cc:407] Connection header received
0812 09:44:56.778314 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:56.778318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:56.778364 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:56.778455 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:56.778752 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.779474 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.780387 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.780613 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.780731 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:56.781088 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:56.781176 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:56.781288 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:56.781440 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 09:44:56.781495 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 09:44:56.781896 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:56.777355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.3.147.61:33217)
0812 09:44:56.777492 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:56.777497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:56.778038 (+ 541us) server_negotiation.cc:407] Connection header received
0812 09:44:56.778171 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:56.778174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:56.778230 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:56.778316 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:56.778601 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.779209 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.780703 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.781028 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.781121 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:56.781363 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:56.781437 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:56.781480 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:56.781677 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 09:44:56.781739 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 09:44:56.782790 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 09:44:56.777299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.3.147.62:39905)
0812 09:44:56.777552 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 09:44:56.777558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 09:44:56.777774 (+ 216us) server_negotiation.cc:407] Connection header received
0812 09:44:56.777927 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 09:44:56.777930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 09:44:56.778083 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 09:44:56.778196 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 09:44:56.778591 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.779420 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.781200 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 09:44:56.781540 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 09:44:56.781642 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 09:44:56.782077 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 09:44:56.782374 (+ 297us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 09:44:56.782484 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 09:44:56.782564 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 09:44:56.782620 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
I20260812 09:44:56.790033 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 29794
I20260812 09:44:56.799531 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 29877
I20260812 09:44:56.807883 3660 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknDSHUD/build/debug/bin/kudu with pid 29966
2026-08-12T09:44:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3176 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6384 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293130 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST